Machine Learning - Data Description - Measures of Positions Standard Score and Outliers

In this session we are discussing measures of position that is standard score and

outliers. So at first we are discussing standard score also known as Z

score. A Z score or standard score for a value is obtained by subtracting the

mean from the value and dividing the result by the standard deviation and the

symbol for a standard score is z and the formula can be visualized in this way that

is z is equal to that very particular value minus mean by the standard division

For samples the formula is x minus x bar by s and for the population the formula will

be x minus mu by this sigma. So here is stands for the standard division for sample and x bar is the mean for the

sample. Similarly, mu is the mean for the population and sigma is the mean, sigma is the

standard division for the population. So, the Z score represents the number of standard

divisions that a data value falls above or below the mean. So, this is the purpose of

the Z score, also known as the standard score. Now, we shall discuss outliers. These

outliers are very important and in our machine learning will be using these terminologies

for multiple different places An outlier is an extremely high or an extremely low data value when compared with the rest of the data values

So procedure to find our outlayers. So here we are having five different steps

Let us go one by one. Addens the data in order and find Q1 and Q2

So Q1 and Q2 are the first and the third quartile. So step two

the interquital range, so it will be denoted by the abbreviation IQR, interquartal range

So that is nothing but Q3 minus Q1. Step 3. Multiply the IQR that is interquartile range by 1.5

Step 4. Subtract the value obtained in step 3, that is our respective IQR into 1.5

So, subtract the value obtained in step 3 from Q1 and add that value with the Q2

Step 5, check the data set for any data value that is smaller than Q1 minus 1.5 into IQR

or larger than Q3 plus 1.5 into IQR. So those values which will be falling beyond this ranges will be known as outliers

So I think for the better understanding latest. Let us see one example

So, example of out layers. So check the following data set for outliers So here we are having a set of data How many data we are having here Here we are having eight data So at first we are calculating the Q1 and Q1 is equal to 9 and

Q3 is equal to 20 here. So now questions might be asked how this Q1 has become 9 and Q3

is equal to 20. So as we are having eight number of data and 8 is an even number, so I cannot

get the middle most data. So to calculate Q2, that is the second quartile, that is the median

of this data I shall have to do the average of data number 4 and data number 5

So, the 4th data is our 13 and 5th data is about 15

So, 13 plus 15 whole by 2, so 14 will be our Q2

So here we are having this 14. So in the first half, in the first half before this median, in the first up we are having

how many data we're having 4 data. So to calculate this Q1, I must be doing 4 data means it is the even

number of numbers. So, I cannot get the middle most value. So, I shall take the average of

this 6 plus 12. So 6 plus 12, that is the average of 6 and 12, rather. So 6 plus 12

what is that? That is our 18 by 2, what is that? That is our 9. So Q1 is equal to

9 here. So as we know that here the 14 is the second quartile, that is a median. So here

on the and above this particular 14 we're having four data even number of data so to calculate

this Q3 I must be getting the average of 18 and 22 so 18 plus 22 hold by 2 so I shall be getting here 20 So 20 is our Q3 and 9 is our Q1 So that is the step one we followed Arrange the data in the order

and find Q1 and Q2. So we have preferred that one. Next we are going for this IQR, that is

interquartile range. So that is Q3 minus Q1, that is 20 minus 9 is 11. So we followed the step 2

is the interquital range we have calculated. Now I shall multiply this value with 1.5

So 1.5 into 11 is equal to 16.5. So step number three, multiply this IQR by 1.5

It has been done. Next. So the lower limit is equal to 9 minus 16.5, that is minus 7.5

And upper limit is equal to 20 plus 16.5. That is about 36 point. So that was ox to be done

here. So, now we are going for this step number five. So, check the data set for any data

values that fall outside this interval, that is minus 7.5 below to that, to 36.5 above

to that, if there is any value existing, then the value will be treated as a, as an outlet

So, here we are getting this value, that is the value 50 is outside this interval, hence

it can be considered as one outlet. So, in this way, we have discussed that how to calculate the standard score and how to determine

the outlets in a set of data in this video. Thanks for watching this