MapReduce and Design Patterns - Chain Folding Pattern Overview

In this video, we are discussing chain folding pattern overview

In this particular pattern, we shall optimize the map phase tasks. So, let us go for some more discussion on this topic

What is chain folding pattern? The chain folding is the optimization level of the job chaining

We know that job chaining job means actually multiple number of map reduce tasks will get

executed one after another in the process of a chain. So, this chain folding pattern will actually optimize this operation

And when there is one large job chain, then we need to use this chain folding mechanism

to combine map phases to optimize the total task. So, whenever a chain is having multiple number of map reduced tasks, then this particular

design pattern can be used so that we can combine the operations to be done in the

map phases so that the process will become as a whole optimized and lesser amount of data

transfer will take place so it reduces the amount of data movement in the map reduce pipeline

so let us give you one example for the better concept so here you can find we are having

two map reduce tasks so there are two tasks are there so comments are coming and then here in this phase in the mapper phase we doing filter out tin Azure comments so it is doing this one then the tin jr tinajure comments will be the outcome and that will be taken as input for the

next map reduce phase so here in this map reduce task we are doing there is a

tokenized remove stop keywords so that is the operation to be done and the reducer

will perform the word count so how many tasks are there in this particular

map reduce change here we are having two tasks are there so there is a tax number one map reduce tax number

one and this is our tax number two so here you can find that we are having used data

flow in this chain its output will be taken as input for this map reduced task and then the final

output will be obtained that is team Azure watt count so now in case of chain folding pattern

we can do this one in this way we are going to do the optimization of the map phase

processes and tasks so comments will be be taken, users are there and then we are having this map filter out teen azure comments

and also tokenize remove stop words and then reduce will be coming the reduce the reducer

the reducer will do the what count and the final teen ageer what count will be obtained

So this is the basic concept and in this diagram we have explained that what is chain folding pattern

So in the next videos we'll be discussing more on this particular topics. please watch all of them and thanks for watching this