Machine Learning - The Learning Process

In this video we are going to discuss the learning process

So, what are the different steps through which our data set will be ready and that

will be given to a model for the learning and that we are going to discuss here

So this is our real world. So the first step will be there from the real world, data gathering has to be done at first

Then we will be going for data pre-processing, dimensionality direction, model learning

and then model testing and then our data will be ready for the ysis of the

unknown data for the prediction so this is the basic steps in our learning

process so at first we are going for the data gathering this data gathering can

be done from different devices from different transactions from camera from sensors from databases and so on so there are multiple different sources from

where we can gather our data but whenever we shall gather our data that data may not be

in a good format so that's why you'll be going for data pre-processing where in this

case we'll be going for noise filtering if you find that some data are corrupted some

data are biased so in those cases this noise filtering has to be done so that we can get

our data without any noise next one we are going for the feature extraction and normalization

After doing this noise filtering, our data will be obtained which is free of noise, but sometimes it may have some missing values

So some of the attributes, some of the fields for a certain row or record might be having no value

In those cases, the missing values are to be resolved. Then we'll be going for the feature extraction

We might be having, say, thousands of attributes, might be having hundreds of attributes

If you go on working with all the feature extraction, we might be having thousands of attributes, if you go on working with all, those attributes the computational complexity will be too high and the model will be

functioning very slow and all those attributes may not be required at all so that's

why we should find out those principal attributes which are required for the

prediction purpose and which will be required for the learning purpose of the

module so this particular model should be learned on those particular set of

attributes which are relevant and principal attributes then the normalization different type of attributes may have different minimum value maximum value The age of a person for the particular database we have found that the age

of a person is ranging from, say, 20 years to 80 years. The height of the person is ranging from

say, 4.5 feet to 6.5 feet. So in this way, we are having multiple different attributes

are they are having different set of or range of data so all these data are to be normalized

are to be scaled so that they can be processed for learning next we'll be going for the

dimensionality deduction here we'll be going for feature selection and feature projection so in case of feature selection and projection we should we are trying to

find those attributes which are relevant for our learning process for our prediction

process also we can assign some weightages on those attributes so attributes with some

higher weight age will be more relevant attributes with the lower weightage value will

be less relevant so these processes are to be done in our dimensionality

reduction and then we'll be going for this model learning we can go for the

classification regression clustering descriptions and so on in case of classifications let us suppose there is one

credit card approval data set where applicants certain personal details are written say their age, their

sex, their income, their savings, say the respective their whether they are having their own house

whether they are having their own car. So all these details are there. And then we are having a past

data which is depicting that whether this particular persons have got the sanction for the

credit cards yes or no so approved and not approved in this way I'm having a set of data

so on that particular set of data I shall make my model learn now when I shall give some new

applicants information to my model the model will tell us whether the credit card should be

approved to him or her or should not be approved to him or her so that is the that is known

as a classification so classification means we're having two classes in this problem

whatever I have discussed, that is approval will be done successfully or not approved

So, approved and not approved. So these are the two outcomes So we be calling them as two classes So what will happen when a new data set will be given then our model will predict whether approved or not approved

So, it will go for a certain class. So that is known as the classification

Whenever we are having our data in a continuous fashion, then we can plot X and Y for

them using some regression and then using some graphical representation and then we can

draw one line through those points which is known as a regression point so that for a new

value of x i can predict a new value for y so in that case the regression can help us for the

prediction purpose we are having this clustering and descriptions so in case of clustering

what we can do we can select those data sets in a certain group which are having some common

similarities or having some distance within a certain bound So, that is known as the clustering

So, in this way, in case of model learning, different types of algorithms can be used

That is the classification algorithm, regression algorithm, having the clustering algorithm, and so on

After making the model ready, then we shall go for the model testing

So here we are going for the cross-validation, bootstrap, and so on

So this is known as the model testing. In the case of cross-validation, what will happen

our data set, a portion of the data set will be used for the training and another portion

of the data set will be used for the testing. So as I am taking our testing tuples from the data set, so we are having our labels and

outcomes known to us. So now I shall check how far my model is doing the prediction correctly

So that is known as the respective cross-validation. Now in case of bootstrap, bootstrap aggregating also called bagging is a machine learning

and simple meta algorithm designed to improve the stability and accuracy of machine learning

algorithms used in statistical classification and regression. So this particular bootstrap is actually a process with the help of which we can increase

the ability and the stability and accuracy of our machine learning for the correct prediction

So, regarding all these aspects in our tutorial we are going to discuss so many different issues

So in one slide obviously all the issues cannot be discussed So but each and every aspect we have discussed in our tutorial with some examples with some implementations and all

So training data, that is our past data, that will be given to this model for the

learning purpose, and now test data, that is our future data, will be given to our model

for the prediction. So gather data from various sources. So that is our step number one

data to have homogeneity. Build model select the right machine learning algorithm which is applicable there and gather

insights from the models results and visualize that is transformed results into some visual graphs

Obviously having data on in a table and having the graphical representation of the data

will be more comprehensive. So transform results into the visual graphs and representation

randomly split examples into training set u and also test set v use training set to learn a

hypothesis h and measure the percentage of v correctly classified by h and repeat for different

random splits and average results so we can we can split our data set randomly into training

set and test set and we can we can go for the performance evaluation and then we can

average those results to decide the efficiency, ability, accuracy of the model

So, overfitting, what is overfitting? The model learns the training set too well, it overfits to the training set and performs

poorly on the test set. So when the model will be tested with our data set which is pre-known to us, it is performing

okay, the training set is on the training set it is performing okay, is very good performance

predicting most of the time correctly but when the new data set will be given

to it it is performing very poorly and in that case we can you can consider

that overfitting is the problem so underfeiting when the model is too simple

and both training and test errors are very large so in this way we have

discussed multiple different issues in this particular video please watch the next video also because that video is also in the continuation of this

particular video so watch the next video where you'll be doing discussing different aspects into further details and what are the three different categories

of machine learning model that will be also discussed. Thanks for watching this