Basics of MapReduce Algorithm

In this video we are discussing basics of MapReduce Algorithm

So, how this Map and the Reduce two tasks with the help of the mapper class and

the Reducer class, how the things are taking place we shall be discussing in this video

with one sample example to clear your all doubts. So, let us start with this basics of MapReduce algorithm

So here we are having the mapper class, also we are having the Reducer class

output of mapper class will be the input to the reducer class the mapper class is

taking the input and ultimately from the reducer class we're having the output

formed and these inputs and outputs will be in the form of key value pairs this

mapper class will do the tokenizing input mapping and shuffle and shot and

here this reducer will do the searching and reducing operations so these are the

main prime operations in mapper class and these are the two operations to be done in the reducer class. So, different tasks in map reduce algorithms

So, let us discuss more into details. The map reduce programs have two tasks, the map task

and the reduced task, as it has been shown here. The map task is done by the mapper class

and the reduced task will be done by the reducer class. So, this is a map task done by the

mapper class Reduce task will be done by the reducer class Mapper class takes an input tokenizes it and maps and shorts it You can find that map class or the mapping class is taking one input tokenizing it

and doing the suffling and short. So the mapper class takes the input, tokenizes it, and maps and shorts it

The output of the mapper class is used at the input to the reducer class, as I have mentioned

earlier, which in turn such as the matching pairs and reduces them, according to the business

logic, whatever is required, the respective outputs will be obtained and respective customized

function will be working at the reducer class for the reducing operation

MapReduce task example. So, one of the most fundamental map reduce example is word count problem

This work on problem is something like our hello world. problem in MapReduce. So it takes the string with different words as input and counts the number

of words in of each type and let's see how the mapper class and reducer class work for this

task. So a string will be taken as input and then it will count the number of distinct words

in that particular string and that is known as a word count problem. The mapper class tokenize

the strings and makes a shorted list of the words and it makes the words. It makes the words

as the key and the number as the value. So, what will become the key and the number

That means the frequency of occurrence that is a number that is a count will be the value in that case The reducer class takes the list and count the number of entries of each word in the input

which is the output from the mapper. So, the reducer class takes the list and counts the number

of entries of each word. It will check that how many times one word has occurred

Finally, creates another list with key and their respective values, that is the count values

is number of times for which the word has occurred in the sentence

So, here is one example for your better understanding. So, here you see, there is one sentence is there

One input is here. That is our a duck is a bird

We have taken a very simple example. So, mapper class, here we're having take line from the input and tokenize it

And for each word in the line, emit word, comma, one. So there is a, it will create such tokens here

For each and every word in the sentence, it will go for emit word comma 1

So, in this way, it will produce output like A, comma 1, duck, 1, is comma 1, A comma 1 and

B, B, comma 1. So that is the mapper output. Next, it is going for the shorted output

So now it has got shorted in the alphabetical order. So, A, A, B, D, I

So in the alphabetical order, on the keys, the corresponding shorted output has been obtained

Now, remove duplicate keys. So here you can find that A has occurred for twice So that why the key A has got removed here and rest of them are unique So no elimination was taking place So now this this output will be going as

the input to this reducer class. So, reducer class is having one algorithm, something like this

So, reducer key, comma, values. So sum is equal to zero. For each value in key, sum is

equal to sum plus value and emit key comma sum. So, in this way, where it is taking the key

values as input and emitting the key sums as output, and that is also a key value pair

So, A will become, we'll have the sum to, bird one, duck, comma one, and we're having

is a final output which will be obtained. So from this example, there is a word count example

it is getting clear what is the purpose, what are the operations are being done in the

mapper class and also in the reducer class. MapReduce task example, let the string is

a duck is a bird, it has five words and the input is given as input to the input of the

mapper class. So, at first this input will be taken by the mapper class. And mapper will generate

the key value pairs. After that, reducer takes it as input and find final word count. And

the output. In this way, the final word count will be obtained as the output. So, in this video

we have discussed what is the basics of MapReduce algorithm with details discussion and

with some sample examples. Thanks for watching this video