MapReduce and Design Patterns - Counting with Counters Example

In this video we are discussing counting with counters example and this video will be

assisted with one practical demonstration to give you the idea how to write the

course run and implement. So counting with counters example. So in this example

we will be using batches.xm. So we are having one xm that is a batches dot XML on which

our program will be working which will count the number of different batches. The

batches are having different categories and there are three types of batches are

there one is gold next one is silver and the last one is the brooch so after

completing the task it will return the count of each batch type so that's

that's the problem we are going to implement now so let us go for one practical demonstration for the easy understanding of this concept under the

summarization design pattern we are going to implement counting with counters it is a

map only job we're having the input file batches dot XML which is under the folder slash input slash

batch so these batches dot XML is having multiple tags are there and multiple attributes

are there so let us go for the the content of this batches dot XML so batches is

one tag under this tag we're having multiple number of rows are there we're having

ID which we are also having the user ID then name then date then class

and then tag based this class is having multiple values and three values are

mainly there three two and one it is a map only job so we are having only one

class in this case and that is a class is counting MR task and as it is a

map only so that why we only extending one inner class from the mapper we having one enumerator here so there is a protected static enum containing gold silver and

bronze so now this is a inner class there is a static class there and this

inner class that is a counting mapper extends mapper so we are not having any

kind of reducer here so we are having only class inner class that is a counting mapper which is extending mapper and which is overriding the method

that is a map so in this particular map we're having one hash map object

XML parsed and this particular object is getting instantiated with the method

that is XML to map which takes one XML as input and produces the hash map object

as output and that will be put onto this XML parsed now with the method XML parsed

we're reading the class value we're reading the class tag value we have

shown you that we're having one field one attribute that is a class in our batches dot XML see this class is

there it is having three values one two and three so here we're checking that

with if the type dot equals one then from the context objects ct x we are

getting the gate counter and the batches batch type countered gold will be incremented by one so here the gold count will be

incremented by one and we know that we're having this type of three values are there one two and three classes are there so for for the

silver we're incrementing the value if the type is two for branch if the type

dot equals is three then the value will be incremented by one so this is my

respective main function here we are checking that how many arguments were

passing the argument should be passed as to if the arguments are not passing as two so then the respective error message will be displayed so this is the error message we having an a system dot exit 2 we are defining one job object and this job

object will have the name count number of batches of different types so we are

creating one job object and job object will have the name count number of

batches of different type we're having the file input format and file output

format so they will be initialized with argument zero and argument one we're

having only mapper so that's why we're just this is a mapper extended class

so that is accounting mapper dot class and set num reduced task zero so

because that is no reducer will work here set output key so output key and output

value so both are initialized with this null writable so both are initialized with

this null writable and here you are going to take the completion there is a

status of the completion so whether it is true whether it is zero or one

depending upon that we can put the respective message and then the system dot exit

code so in this way the main function the main method has been written here

the respective number of types we are printing that is how many types of

this gold this bronze and silver so their respective count will be also printed

using this for loop so at first we shall go for the creating the jar so we shall go for this

and then then the respective path name and the file name has to be given already we have

created the jar file so we are leaving the scale steps but after that we will be going for next and finish now let me execute my command so command is Hadoop jar and the path in which the jar file will be

existing so that is a map produced design pattern jar files so this is a path

in which the summarization pattern dot jar this jar file will be existing then

counting counter is the respective the respective package name and under that

we'll be having counting a M.R. task. So there is a class name. Input slash input slash badge

is the input file path name and slash output will be the output file path name. So in this way

we are going to execute our command. So let us execute the comment. We can find here that is

the bytes written is equal to zero. The bytes written is equal to zero means it is only

it is a map only job so nothing will be written by the reducer but you'll be getting the

output in the last three lines there is a number of gold batches 5-2-3-7 number of silver

batches are given number of bronze batches are given here so you can find that the

respective batch count we have also got it and we have printed that one because we had

that fall loop at the end of the Java code so from there the batch counts

respective three classes have got printed so now let me come to the output folder

so here is the output folder and you can find that we are having two part files

but both of them are having the size of zero bytes two part files are there

having got the size of zero bytes because nothing has been written onto the

part files so let us delete them let us delete them so DFS S DFS DFS

DFS minus R and this is the output path is getting deleted. Thanks for watching