0:00
Hi thereof friends welcome back here in this lesson you are going to learn about
0:07
optimization techniques till now we have learned about cost function activation function and a lot of other things which are related to the neural network now we
0:18
know that neural network consists of interconnected nodes called neurons okay and
0:23
they are organized into layers now each neuron has some weights and
0:29
some biases which will go to determine its behavior. Okay, doing, you know, whenever we're going to build a new network, we need to train that
0:39
neural network with some labeled train data sets. Okay. Now at the initial point, we put a
0:46
random value to the weights and to the biases and as soon as the training process
0:52
started, starts and it will start adjusting the value of these waves and the biases
0:58
That is why they are also known as the learning parameter because we are adjusting the values of weights and the values of the biases
1:06
And why we are doing it so that the value which are predicted by the neural network is somehow closer to the actual value
1:14
Okay. And the difference between the actual value and the productive value is known as the cost functions, which we have already learned about it
1:23
Okay, now suppose we have learned about the activation function, okay, which, which is known
1:28
which will introduce the non into our neural network output Now suppose this is the graph is as you can see on my screen it is a U graph this is a perfect example for the non graphs and now as I said that we going to put some random values to our weights and the biases
1:50
suppose it is over here now we need to make it to come over here this point where you can say it as the
2:00
local minima where we're going to have very less value of the cost function it is
2:08
where your predicted value the value coming from the neural network is somehow
2:12
near is somehow similar to the actual value so how we need to do that you're
2:19
going to use this optimization technique this gradient is the way in order to
2:24
minimize the cost function so we're going to take some small steps okay and
2:29
And this small steps are basically known as the learning rate. Sometimes it will be bigger, sometimes it will be smaller
2:37
It will adjust accordingly. Okay, now it doesn't know that where is the local minima is there
2:44
So if there might be situation, you are going from this direction
2:48
We are decreasing the value, decrease the value, and then suddenly the value started increasing
2:53
The difference, you know, is starting increasing. So we need to adjust our learning parameter in such a way that it can
2:59
Converse to the local minima. So this is how it works. Okay, so this is the
3:06
example which you have to understand that how this is a gradient is and Dissent works how this optimization techniques works So basically optimization technique is basically we going to optimize the new network The adjustment of the learning parameters are called optimization techniques
3:23
Now we have multiple automation techniques. Let me just describe three of the most popular optimization technique
3:32
The very first one is stochastic optimization technique. Then we have the batch optimization technique and then we have mini batch optimization technique
3:41
These three are very much popular so in the case of the batch optimization techniques what we are going over here see and
3:48
There is something which makes different to other optimization optimization techniques So in the case of the batch
3:57
optimization technique we are passing the whole data set all together and once all the data set are evaluated by our neural network and
4:01
once all the data set are evaluated by our neural network and
4:05
then it will going to adjust the value of the weights and the biases but in the case of the stochastic
4:11
what will happen after every iteration if we're going to adjust the values of the weights and the biases
4:17
so both are doing in the different order okay so here in the case of batch way the batch
4:26
technique it will going to take long period of time whereas in the case of stochastic it won't take so much time
4:32
It would be somehow faster as compared to the batch optimization technique
4:38
But you know this batch is very much helpful if you have a very small data set because if it is you know small it will be processed the data and adjust the value very fastly so you can consider another
4:54
variant of this batch which is mini batch where instead of putting whole
4:59
data set whole training data set into one go we're going to you know choose a
5:06
batch size and that batch size will go in to do the same thing which we are
5:11
doing with the batch optimization technology. So one after that batch is completed, then we're going to adjust the values of the learning parameter and make our Neonetwork more accurate
5:22
Just reducing the cost function. So this is the whole process how it works. Okay, so I hope it is clear to you that
5:30
What the relationship between the optimization technique and the cost function? You might be knowing that or not that when I was discussing about cost function. I have narrated that that
5:40
that cost functions guide the optimization technique in order to make the neural network more accurate
5:47
okay so they are somehow very very connected to each other okay so this is all about
5:56
the process of optimization and the cost function and the activation function
6:03
how all they work together so in the next lesson we're going to learn about different
6:08
types of optimization techniques how to implement them. Here we have understand the concerts in the next video we're
6:16
going to implement those optimization techniques. So till then keep learning, keep
6:21
exploring and stay motivated. See you
