MapReduce and Design Patterns - Installing Hadoop on Pseudo Distributed Mode

In this video we are discussing installing Hadoop on pseudo distributed mode

We shall show you and we shall discuss step by steps for your easy understanding and also

this video will be assisted with one practical demonstration and we shall show you that

how the steps are to be carried out. So let us start discussion on this installation process

Installing Hadoop in pseudo distributed mode to set up Hadoop for pseudo distributed mode

set up the following XML files located in this folder. You see, this is a path is there

and there is a folder under this particular folder where having these three files are there

Multiple XML files are there, but we are concentrating only on these three files

And each and every file where we'll be putting some set of XML

So, first one is the core hyphen site. dot XML, SDFS, hyphen site

dot XML, and map rate, hyphen site. dot XML. So, three XML files are there and they require some update. So, set up core

hyphen site.com. That was the first in our list. We are going to update that one

So open core hyphen site. XML from the given path. You see, we had this part and

this is the respective file name here and put the following lines in the XML file. So you can

open this file. It is a text file. You can open it using G-Edit and then put this file, put this

lines onto the respective file. Next was HDFS hyphen site.xml, which will be found in this respective folder

And this is the code's, XML codes, are to be written into the file

So, open the file using G-Edit or any other text editor and put this XML code there

And the last one, there is a MapRed, hyphen site. It will be found in this respective folder

and if the file is not present, copy the MapPrette site. Dot XML dot template

So, this is a file will be there. So, there is a Maprette-Side

Dot XML dot template file and rename the copied file with the MapRet-Hifensite

So, if the file is not directly there, you will be having one template file

So just make a copy of that and rename it to this name

And put the following lines, which are written in XM. to that file itself

Open the hadoof hyphen env. . . So, this is the path from where you are supposed to open this shell program

The shell program is there and export the Java underscore home environment variable in that file

So, how to do that one? Put the following lines there. There is an export Java-h-home

There is a underscore here. export Java underscore home is equal to slash user slash local slash Java slash JTK 1.8.0

underscore 171. So, just see your path in my system the path was like this, whatever I followed the steps

My path was this but if you're having some different version of Java, then obviously the

respective export the string will be also changed. So export Java underscore home is equal to so this line has to be put in the

respective file, there is a Hadoop-env.env. So, I think it is better. Let us go for one demonstration

to show you that how the steps are to be carried out and what are the different ways to do

that one for your better understanding. So, here is the demonstration for you. So, in this video

we are going to discuss how to install Hadoop version 2.4.1. So before installing Hadoop version

2.4.1 we are going to install SSH and R Sync. So what is SSH? SSAH is nothing but

secure shell is it is a secure shell and it is a cryptographic network

protocol for operating network services over an unsecured network and that is

the purpose of SSH. Now what is our sync? So R sync is nothing but is a

utility for efficiently transferring and synchronizing synchronizing synchronizing synchronizing files across computer systems by checking the timestamp and the size of the file

And this R is having the full form that is a remote sync And here also we going for the setup for SSH without pass phrase now actually this pass phrase is actually this concept is required for the additional security

here so to install SSH the command is pseudo APT hyphen get install SSH to

install R sync the command is pseudo APT hyphen get install R sync so we'll be

issuing these commands in the terminal prompt and setup for SSH without pass phrase is the command will be SSH

hyphen t jen hyphen t r s a say and while installing this we'll be

pressing enter for three times to ignore the next stages so let us go for the

demonstration that how we are doing going to do this one so at first we are

going to install this hardu version 2.4.1 highlighting this one and now we are

going for the first command that is a installation of SSH going to the terminal

clearing it up so now here we are issuing the command pressing enter the command

is getting executed yes installation has been done yes the installation is still

in process and it has been done successfully now we are going to copy this pseudo

apit hyphen get install r sync so this is a command we are going to issue now

So we are doing the copy and then coming to this paste and the command prompt has been obtained

Now we are going for SSH without pass phrase. So copying this command here and you shall press three times enter you can find that and

here we are pressing enter three times. You can find that we have entered three times and here we have got we have got back the prompt

So that means the instruction has been done successfully. Now here we are copying this particular file and actually we're concatenating it at the end

of the file that is the slash dot SSH slash authorized underscore keys

So the left and side file will be copied at the end of this right-and-side file

If the file on the right-and-side does not exist then the file will be created and the new content

will be pasted there. So this is a command we are going to issue now. So let us go for this copy and issue this command

comment has been issued you can find that now we shall come back we shall issue

the command that is our SSH local host so going for this marking SSH local host

we shall go for yes pressing enter so the prompt has been obtained now here

we're having some steps are there so we'll be here downloading this Hadoop

version 2.4.1 from the link link as it has been given it we shall do the copy and paste onto the URL box so

let us go for that operation and the Hadoop will get installed on our system copying

this URL then opening the browser and go for paste and go now you are going

to select the 132 mb file that is the Hadoop 2.4.1.tard dot gz going for save the file

is getting downloaded on our download folder you can click here you can see the

progress how the file is getting downloaded here so now it is almost

getting completed the download has been done onto the download folder opening the folder now so we are creating one folder under home the name of

the folder will be hadoop giving the name and now all the

extracted files from the zip file we've done the dragging it so that all the files will get copied onto our Hadoop blank folder

Shall close this one. So this is our Hadoop folder. So we have followed the step that is the extract that downloaded files into the Hadoop folder

So this much we have done. Now we are going to edit our file that is a slash ETC slash profile file

And the following two lines are going to be added at the end. end of the file that is a profile file which is residing under the ETC folder so let us go for

opening this there is a 80 slash it is slash profile file so let us go for uh opening so pseudo g edit going for slash it is slash profile opening the file it is asking for the password giving the password here cannot open I think we should

terminate this terminal let us go for another one let us open another terminal

here okay so another terminal has got open we're going for this pseudo let me

execute this one again so pseudo g edit slash etc slash profile

pressing enter giving the password so file has got opened we're coming at the end

creating some space here now coming to this install and copying these two

lines so here we are exporting this Hadoop underscore install this variable is

getting initialized with this path and also export path so is equal to dollar

path colon Hadoop Hadoop underscore install slash bin so these two these two lines were just doing the copy and paste at the end of

it is slash profile file so doing the paste so file has got updated now saving it

then I shall close save and close then I shall execute the ETC profile file so

that the respective changes will get effective in the system Okay, we have done it

So execute the slash ETC slash profile file. So this much you have done

Now Hadoop pseudo distributed mode. So we'll be going for some updates in the XML files and so on

So let me go show you. So at first we are dealing with this core hyphen site.xmL

So this is a file where we are going to do some updates and this file is residing under

the folder that is a Hadoop slash ETC. It is C slash Hadoop slash core hyphen site.xml

So this is a file is there where we'll be pasting this configuration portion

The portion which is written in this configuration tag. So let me open this core hyphen site

So it is under this Hadoop. It is C, Hadoop. And here we're having this core C

yes see is the starting character so I should be going for core hyphen site dot

XML so opening this one here you can find that this configuration tags are blank

so let me copy this one at first so I'm done the copy and then I shall go for

this core hyphen site dot XML we have done the paste you can find that only

we have just updated the portion we're not touching any other portion of this

file just updating that portion whatever you wrote you can pause the video we can do the same you can make this x-m-l ready and then

we are saving and closing it so we have done this code hyphen site dot XML so required

update we have done now we are going for the next one that is a hadoop slash etc slash

hadoop slash hdfs hyphen site dot XML so that is another XML file is there just remember

remember this path here so hadoop slash etc slash hadup slash sdfs hydecddfs hydexml so these are xml

we're going to do some updates we are going to carry out some updates on this particular file

here so what is update so here this configuration portion has to be written there so let me

open the open the file now so doing the copy here so sdfs the file is already open so that is

the sdfs hyphen site dot XML so here you have done the copy and paste here so

configuration tags have got updated so here we are having the replication

factor of one we can also go for three so here we have gone for the

replication factor dfs dot replication so this particular property has been set

with the value one here okay so we have done done it now you shall go for the

save and then you shall go for the close next we're going for

the next file which is the map red hyphen site.xml which is residing under the folder that is the Hadoop slash ETC slash Hadoop

map red hyphen site so this is a required file where this configuration section has to be updated so here you are putting this map red

job dot tracker a local host colon 9001 so name and the value so they

have they have got enclosed with the respective tags so we're going for this so

here can find that we are not finding any map red hyphen site dot xm

we're not finding this one so what we have finding we're getting map red hyphen site.xml dot template so what we shall do we

shall make a copy and paste so we shall make a replica of this template file and then

we shall change the name to map red hyphen site dot XML and there will be doing the

required updates in the tags so let me make a copy so I'm done the copy and paste

so it has made a paste here so now I shall go for this template and this blank

space is to be eliminated yes we have made the thing so now coming to

this installation details.txte taking this configuration portion okay now going for this map redside.xml this configuration portion is getting updated with the new content

you can find that here we have done the copy and paste so please remember all

these steps because all these steps are necessary and they are to be done the order

in which you are demonstrating here so now we shall go for this save and exit

okay now we're going for this Hadoof env Hadoop hadoop hymn.sh so here we are supposed

to put the Java path here Java home path so this is a Java home path we are

getting so it does doing the copy don't take that colon in the selection so

here here in this case we're just pasting the required Java home path here so

you can find here so actually this is a put the Java home

actually i should write this one as java home at hadup hyphen env dot sh so this particular shell program

has to be updated with the java home path so we have done that one and we have got the java home

by just writing the java home dollar java home we have written that one so getting the path

and we have done the pasting there now run the final command to set the set up the haddub okay so we'll be

going for this name note format will be going for the name note format so let us go

for that section now so yes we have done this now we shall go for this name

note format so copying and pasting it here so name not format is taking place

yes it has got pasted so formatting has been done so let me write this

lines how to start the Hadoop so we're supposed to issue it should be

written in capital letters so dollar Hadoop underscore install slash sbin slash

start hyphen all dot SH so this is the there's a shell program you are supposed

to execute you can find that Hadoop is now is getting initiated as we'll go for yes so we're going to see what are the processes are running

so here you can find that node manager data node name node secondary name node we're

having the resource manager so this these are the respective processes are

running in the background so that means my hardube is on now let me open this one in the browser so the command is

local host colon 50070 so if you just issue this one in the browser you

can find that this particular hadoop has got so respective GY view we're

getting here so in this way we have shown you that how to install this

Hadoop here. So in the browser we are going to write local host colon 50070 to get the

Hadoop overview page on. So throughout this demonstration we have shown you that how to install

hardup on our system. Thanks for watching this video