0:00
Hello friends, welcome back
0:04
Here in this lesson you're going to learn about activation function. What they are, why do we need them, and also different types of activation functions
0:12
So here we go. So what basically this activation function means? Activation function is a mathematical function that introduces the non-linearity into the
0:22
neural networks output and it is typically applied to the weighted sum of the input of the neurons
0:28
and it determines the output which means that our neuron will be activated or not activated will be totally dependent upon the activation function
0:36
So let us revisit once again So basically the purpose of activation function is to add the non linearity
0:43
So that it can learn and recognize a complex pattern from the data and build our neural network in such a way
0:52
It can predict the future values. Okay, now without activation function the Neuronor
0:58
network would be simply a linear equation. linear combination of input values
1:04
So it will going to hinder the learning ability of the neural network
1:07
So having the activation function is very much required. Now you might be thinking, I am taking this term again and again, what this linear
1:15
and non-linearity, what basically means? Okay, let us understand in the mathematical context first
1:21
See in the case of linear the linear relationship between two variables which means if something happened to variable A suppose in case it will be going to happen something with the variable B
1:33
Okay, if you're going to change the value of A, the value of VV also affected
1:38
Okay, and this kind of relationship can be, you know, described in a straight line in a graph
1:43
And here we have the mathematical equation for it. Y equal to Mx plus B. So here, Y is the intercept and the M is the slow
1:51
of line okay this is the case of the linear relationship now what will be
1:56
happen so we're going to have the non linearity into our data see just
2:00
understand it is just opposite of linear here we don't have any straight
2:05
line here it is not dependent on any other variables value okay so it means
2:11
there will be no single you know a mathematical equation would be there
2:16
there might be you know some different mathematical equation will be used for different kind of shapes shapes shapes
2:21
It could be rely, it could be exponential graph, it could be an oscillator graph or it could be a decay graph of anything
2:28
Okay, so which means this having this non-inarity is a very important thing for building our neural network
2:36
Okay, now let us just understand that why we do we need them
2:40
Okay, now just take us a real example. Like if you want to build object detection or speech recognition or language processing kind of application here it understands Here it involves highly nonlinear patterns okay and if you just going to use the linear functions it not going to help you out okay
2:59
in order to make your robust neural network in order to work perfectly you need to introduce this
3:05
this nonlinear functions into this in the process of learning okay now this is the one
3:13
only one not this is not only one thing okay we could have some other things as well like if you're going to put our activation function as linear
3:21
okay and then it derivative will be also linear which causes our cost to gradient flow okay and this can lead to be having a vanishing grading problem where the
3:33
the gradient function will be small okay and it's going to hinder the landing process which means
3:40
overall this activation function is very much important while creating or you can
3:46
say by building the neural network okay so now you may understand what exactly
3:52
this activation function is and why do we need them now let us understand the
3:57
different types of activation function which we have so we have sigmoid activation
4:01
function then we have hyperbolic tenor activation function then we have rectified
4:08
linear unit activation function in which is short we say ReLU RELU okay and then we also have Leaky Relu which is the variant of ReLU activation function only Okay so in the case of Relu we just going to have the value between 0 and 1 and rest negative value will be ignored
4:27
But in the case of Leaky Relu, we're going to have some negative value as well
4:31
We're going to multiply with the alpha value. We're going to discuss each and everything that how it looks like, how you can represent it mathematically and how to even, you know, use them using libraries like
4:43
Numpi and the tensor flow we're going to perform this all these things okay in our
4:49
upcoming videos okay and then we also have the soft mix activation function which is
4:54
basically used in multi-class kind of data sets the sigmoid is used for you know the binary
5:00
classification see we have multiple activation function but we can use this activation
5:05
function regarding what kind of data set you have what is your problem okay then then
5:11
you will be able to choose those activation function and it will going to make a neural network
5:17
more accurate more predictable because it is something which is required when you are building a
5:22
neural network it is something which is required during the training process of that neural network
5:27
so that it can predict a right value so this is all about activation function in upcoming lessons
5:32
we're going to learn how to implement them till then keep learning keep exploring and stay
5:37
motivated see you soon
