MapReduce and Design Patterns - Top Ten Pattern Overview

In this video we are discussing top 10 pattern overview

In case of top 10 pattern overview from the huge data set we are selecting a subset that

is a K number of records in the output data set. Here the value of K is equal to 10

So now let us discuss more on this concept. So what is top 10 pattern

This pattern is used to retrieve relatively small number of K number of records

the value of k is equal to 10. And in this case, we know the exact number of records after

completing the whole task, but the number of records in the final outcome will not depend

on the total records in the input data set. So here we are going to select only the K number

of records. So, that means how many records are there in the initial input splits? So for that

point, it is irrespective of that. To design this pattern, we can split records into different

parts and from each part we will get the top 10 records individually and after that the

reducer will get the final top 10 from those splits and thus this pattern will work so here we are having the mapper and both the reducer So mapper will produce local top 10 records and then those local top 10 records will be made available to the reducer

and reducer will make the final top 10 records. And that is a concept here. So in this particular

design pattern, we require both the mapper and the reducer tasks. So now let us go for one

diagram for the better understanding. So top 10 pattern structure, This pattern uses the comparator functionality to compare each two records

So, here you can find here we're having the input splits. So, we're dealing with the huge data sets

So that will be divided into multiple splits. And then they'll be assigned to the top 10 mapper

So multiple mappers are working. And depending upon the input split, they are producing their local top 10 records there

And this outcome of this mapper will be made available to the top 10 reducer

Reducer will make the final top 10 and that will be the final output of this design pattern

So in this way we have discussed what is top 10 design pattern and in this case you can find

that we have given you the overview and in the next videos we'll be going for its more discussion

and implementation. Thanks for watching this video