MapReduce and Design Patterns - Counting with Counters Example

In this video we are discussing Counting with Counters example

And this video will be assisted with one practical demonstration to give you the idea how to

write the course, run and implement. So counting with counters example. So in this example we will be using batches.xm

So we are having one XML that is a batches dot XML on which our program will be working, which

will count the number of different batches. The batches are having different categories and there are three types of batches are there

One is gold, next one is silver and the last one is the brooch

So after completing the task, it will return the count of each batch type

So that's the problem we are going to implement now. So let us go for one practical demonstration for the easy understanding of this concept

Under the summarization design pattern, we are going to implement counting with counters

is a map only job we're having the input file batches dot XML which is under the

folder slash input slash batch so these batches dot XML is having multiple

tags are there and multiple attributes are there so let us go for the the

content of this batches dot XML so batches is one tag under this tag we're

having multiple number of rows are there we're having ID which we are also

having the user ID then name then date then class and then tag based this class is having multiple values and

three values are mainly there three two and one it is a map only job so we are

having only one class in this case and that is a class is counting MR task

and as it is a map only so that why we only extending one inner class from the mapper we having one enumerator here so there is a protected static enum containing gold silver and

bronze so now this is the inner class there is a static class there and this

inner class that is a counting mapper extends mapper so we are not having any

kind of reducer here so we are having only class inner class that is a

counting mapper which is extending mapper and which is overriding the method

that is a map so in this particular map we're having one hash map object

XML parsed and this particular object is getting instantiated with the method

that is XML to map which takes one XML as input and produces the hash map object

as output and that will be put onto this XML parsed now with the method XML parsed

we're reading the class value we're reading the class tag value we have

shown you that we're having one field one attribute that is a class in our batches dot XML see this class is

there it is having three values one two and three so here we're checking that

with if the type dot equals one then from the context object ct x we are

getting the gate counter and the batches batch type countered gold will be incremented by one so here the gold count will be

incremented by one and we know that we're having this type of three values are there one two and three classes are there so for for the

silver we're incrementing the value if the type is two for bronze if the type

dot equals is three then the value will be incremented by one so this is my

respective main function here we are checking that how many arguments were

passing the argument should be passed as to if the arguments are not passing as two so then the respective error message will be displayed so this is the error message we having an a system dot exit 2 we are defining one job object and this job

object will have the name count number of batches of different types so we are

creating one job object and job object will have the name count number of

batches of different type we're having the file input format and file output

format so they will be initialized with argument zero and argument one we're

having only mapper so that's why we're just this is a mapper extended class

so that is accounting mapper dot class and set num reduce task zero so

because that is no reducer will work here set output key so output key and output

value so both are initialized with this null writable so both are initialized with

this null writeable and here you are going to take the completion there is a status of the completion

so whether it is true whether it is zero or one depending upon that we can put

the respective message and then the system dot exit code so in this way the main

function the main method has been written here the respective number of

types we are printing that is how many types of this gold

this bronze and silver so the respective count will be also printed using this

fordook so at first we shall go for the creating the jar so we shall go for

this and then then the respective path name and the file name has to be given

already we have created the jar file so we are leaving the scale steps but after that we will be going for next and finish now let me execute my command So command is Hadoop jar and the path in which the jar file will be existing

So that is a map produced design pattern jar files. So this is a path in which the summarization pattern.jar, this jar file will be existing

Then counting counter is the respective package name. And under that we'll be having counting a mart task

so there is a class name input slash input slash badge is the input file path name and slash

output will be the output file path name so in this way we are going to execute our command so

let us execute the command we can find here that is the bytes written is equal to

zero the bytes written is equal to zero means it is all it is a map only job so nothing

will be written by the reducer but you'll be getting the output in the last three

lines there is a number of gold batches 5-2-3-7 number of silver batches are

given number of bronze batches are given here so you can find that the

respective batch count we have also got it and we have printed that one because

we had that fall loop at the end of the Java code so from there the batch counts

respective three classes have got printed so now let me come to the output folder

So here is the output folder and you can find that we are having two part files, but both of them are having the size of 0 bytes

Two part files are there having got the size of 0 bytes because nothing has been written onto the part files

So let us delete them. Let us delete them. So DFS, SDFS, DFS minus R and this output path is getting deleted

Thanks for watching