MapReduce and Design Patterns - Binning Pattern Overview

In this video we are discussing binning pattern overview

It is one kind of partitioning pattern. And here in this case we'll be having only the mapper, there will be no combiner

no partitioner, no reducer. So here the dataset will be divided into multiple different output files and one bin will

have multiple output files. So now depending upon the criteria for which this mapper will iterate through the data

it will send the respective record to the respective output file depending upon the matching of

the criteria. So this is the basic concept of this Beaning Pattern overview

So let us discuss more on it. So what is Beaning pattern

It is also like the partitioning pattern and it separates the data into different categories

and but not care about the orders. So the depending upon the category, depending upon the criteria, the records will be

be divided into multiple different output files but it will not bother for the ordering of those

records. So in the binning pattern it splits the data into map phase because here we're having

only the mappers. Now each mapper will have separate files per output bins. So a mapper

can will produce multiple output files and obviously that is a problem. If the system is having

millions of such mappers then so many output files will get generated and obviously that will be a challenge for the name node scalability So if the number of the mapper is large enough it can generate millions of such

output files, so this will create a bad effect on the name node scalability. So these are the

problems with this vining pattern. Now, binning pattern structure. So the mapper iterates

through the list of different criteria for each bin and when the record matches it will

be sent to the respective bin. So it will search for the criteria and list of criteria will be there and it will go on

iterating on those criteria and then depending upon the match, the record will be sent

to the respective bin. No combiner, partitioner, reducers are used in this pattern

So from this very diagram you can easily find that here we are having only the bin

mappers which are taking this inputs from the input splits. So here we are having multiple input splits and multiple billing mapper are working at the

same time in parallel and they are producing multiple output files. And obviously when the number of mappers will be millions of such and whenever we are having

such huge number of output files that will be a challenge for the name node scalability

So in this particular video we have given you only the beening pattern overview

In the next video we will be going for the implementation of the same. Please watch also the next video for you better understanding

Thanks for watching this