Installing Hadoop on Pseudo Distributed Mode

In this video we are discussing setup Hadoop on pseudo distributed mode

So how to do this setup. So there are certain steps that we'll be discussing all these steps one by one for your

better understanding. So let us start the installation process. Installing Hadoop in pseudo distributed mode to set up Hadoop for pseudo distributed mode set up

the following XML files located in this folder. You see this is a path is there

and this is a folder under this particular folder where having these three files are there

multiple xml files are there but we are concentrating only on these three files and each an eb

file where we'll be putting some set of xml so first one is the core hyphen site dot xml

sdfs hyphen site dot XML and map rate hyphen site dot xml so three xml files are there and they

require some update so set up code 5.6. That was the first in our list. We are going to update that one. So, open core

hyphen site.coml from the given path. You see, we had this path and this is the respective

file name here, and put the following lines in the XML files. So, you can open this file

It is a text file. You can open it using G-Edit and then put this files, put this lines

onto the respective file. Next was HDFS-S-site. which will be found in this respective folder

And this is the code's, XML codes are to be written into the file

So, open the file using G-Edit or any other text editor and put this XML code there

And the last one, there is a MapPread hyphen site. .xml. It will be found in this respective folder

And if the file is not present, copy the MapPret site dot XML

So, this is a file will be there. So, there is a MapPrette, hyphen, site

XML dot template file and rename the copied file with the Maprate hyphen site

So, if the file is not directly there, you will be having one template file

So just make a copy of that and rename it to this name and put the following lines which are written in XML to the file itself

Open the Haduf-env-env. dot SH from the given path. So, this is the path from where you are supposed to open this shell

program. The shell program is there and export the Java underscore home environment variable

in that file. So, how to do that one? Put the following lines there. There is an export

Java-hifen home. There is a underscore here. Export Java underscore home is equal to slash

user slash local slash jdk 1.8.0 underscore 171. So, just see your path in my system

the path was like this, whatever I followed the steps. My path was this. But if you're

having some different version of Java, then obviously the respective export, the string

will be also changed. So, export Java underscore home is equal to. So this line has to be put

in the respective file. There is a Hadoop hyphen env. So, I think it is better. Let us go for one demonstration to show you that how the steps

are to be carried out and what are the different ways to do that one for your better understanding

So here is the demonstration for you. So in this video, we are going to discuss how to install Hadoop version 2.4.1

So before installing Hadoop version 2.4.1, we are going to install SSAH and R-Sink

so what is ss h is nothing but secure shell is it is a secure shell and it is a

cryptographic network protocol for operating network services over an unsecured network and that is

the purpose of ss now what is our sync so our sync is nothing but is a utility for efficiently

transferring and synchronizing synchronizing files across computer systems by checking the timestamp

and the size of the file and this R-sync is having the full form that is a remote

sync and here also we going for the setup for SSH without pass phrase now actually this pass phrase is actually this concept is required for the additional security here So to install SSH the command is pseudo APT hyphen get install SSH to install R sync the command is pseudo APT hyphen get install R sync So we be issuing these commands in the terminal prompt and set up for SSH without pass phrase is the command will be SSAH

hyphen key jen hyphen t r s a and while installing this we'll be pressing enter for three times

to ignore the next stages so let us go for the demonstration that how we are doing going to do this one

so at first we are going to install this hardu version 2.4.1 highlighting this one and now we are going

for the first command that is a installation of s-sh going to the terminal clearing it up

so now here we're issuing the command pressing enter the command is getting

executed yes installation has been done yes the installation is still in

process and it has been done successfully now we are going to copy this pseudo

APT hyphen get install R sync so this is a command we are going to issue now so

we are doing the copy and then coming to this paste and the command prompt has been

obtained now we shall we are going for SSH without pass phrase so copying this

command here and you shall press three times enter you can find that and here

we are pressing enter three times you can find that we have entered three times

and here we have got we have got back the prompt so that means the instruction has

been done successfully now here we are copying this particular file and actually we're concatenating it at the end of the file that is the slash

dot ssh slash authorized underscore keys so the left-and-side file will be copied at the end of this

right-and-side file if the file on the right-hand side does not exist then the file will be created

and the new content will be pasted there so this is a command we are going to issue now so

let us go for this copy and issue this command so comment has been issued you can find that

now we shall come back we shall issue the command that is our SSH local host so going

for this marking SSH local host we shall go for yes pressing enter so the

prompt has been obtained now here we're having some steps are there so we'll

be here downloading this Hadoop version 2.4.1 from the link as it has been given it

we shall do the copy and paste onto the URL box so let us go for that operation and the

Hadoop will get installed on our system copying this URL then opening the browser

and go for paste and go now you are going to select the 132 mb file that is the Hadoop 2.4.1

dot tardt going for save the file is getting downloaded on our download folder you

can click here you can see the progress how the file is getting downloaded here so

now it is almost getting completed the download has been done onto the

download folder opening the folder now so we are creating one folder

under home the name of the folder will be Hadoop giving the name and now all

the extracted files from the Jif file we've done the dragging it so that all

the files will get copied onto our Hadoop blank folder. Shall close this one

So this is our Hadoop folder. So we have followed the step that is the extract that downloaded files into the Hadoop folder

So this much we have done. Now we are going to edit about a file that is a slash ETC slash profile file and the

following two lines are going to be added at the end of the file. that is a profile file which is residing under the ETC folder so let us go for opening

this there is a ET is slash ITC slash profile file so let us go for opening so pseudo

g edit going for slash ETC slash profile opening the file it is asking for the password giving the password here cannot open I think we should terminate this terminal

let us go for another one it has open another terminal here okay so another

terminal has got open we're going for this pseudo let me execute this one again

so pseudo g edit slash ETC slash profile pressing enter giving the password

so file has got opened we're coming at the end creating some space here now coming to

this install and copying these two lines so here we are exporting this hadoop

underscore install this variable is getting initialized with this path and also

export path so is equal to dollar path colon dollar had doop underscore install

slash bin so these two these two lines were just doing the copy and paste at the end of

etec slash profile file so doing the paste so file has got updated now saving it then i shall close

save and close then i shall execute the etc profile file so that the respective changes will get

effective in the system okay we have done it so execute the slash it is slash profile file so this much you

have done now Hadoop pseudo distributed mode so we'll be going for some

updates in the XML files and so let me go show you so at first we are

dealing with this code hyphen site dot XML so this is a file where we are going

to do some updates and this file is residing under the folder that is a Hadoop slash

Etc slash Hadoop slash core hyphen site.xml. So this is a file is there where we'll be

pasting this configuration portion, the portion which is written in this configuration

tags. So let me open this core hyphen site.comel. So it is under this Hadoop

It is C, Hadoop. And here we're having this core C. Yes. C is the starting

character so I shall be going for core hyphen site dot XML so opening this one here you can find

that this configuration tags are blank so let me copy this one at first so I'm done the copy

and then I shall go for this core hyphen site dot XML we have done the paste you can find that

only we have just updated the portion we're not touching any other portion of this file

just updating that portion whatever you wrote you can pause the video we can

do the same you can make this xml ready and then we are saving and closing it

so we have done this code hyphen site.xm. So required update we have done. Now we are going

for the next one that is a hadup slash etc slash hadoop slash hdfs hyphen site.xml

So that is another xml file is there. Just remember this path here. So hadoop slash etes

slash Hadoop slash sdf sdf side dot XML so these are xml file we're going to do some

updates we are going to carry out some updates on this particular file here so what is

update so here this configuration portion has to be written there so let me

open the open the file now so doing the copy here so sdf s the file is already

open so that is the sdfs hyphen site dot XML So here you have done the copy and paste here so configuration tags have got updated

so here we are having the replication factor of 1 we can also go for 3 so here

we have gone for the replication factor DFS dot replication so this particular

property has been set with the value 1 here okay so we have done done it now you

shall go for the save and then we shall go for the close next we're going for the

file which is the map red hyphen site.xml which is residing under the folder

that is the Hadoop slash ETC slash Hadoop map red hyphen site so this is a required file where this configuration section has to be updated so here you are putting this map red dot job dot tracker and local host

colon 9001 so name and the value so they have they have got enclosed with the

respective tags so we're going for this so here can find that we are not finding any

map red hyphen site dot XML we're not finding this one so what we are finding we're

getting map red hyphen site.xml. template so what we shall do we shall make a copy and paste

So we shall make a replica of this template file and then we shall change the name to map red

hyphen site dot XML and there will be doing the required updates in the tags

So let me make a copy. So I'm done the copy and paste. So it has made a paste here

So now I shall go for this template and this blank space is to be eliminated. Yes, we have made the

thing so now coming to this installation details.txt taking this configuration portion

okay now going for this map redside.xml this configuration portion is getting updated with a new

content you can find that here we have done the copy and paste so please remember all these steps

because all these steps are necessary and they are to be done the order in which you are

demonstrating here so now we shall go for this save and exit

okay now we're going for this hadoof env hadup hadoop hymn.sh so here we are supposed to put the

java path here java home path so this is a java home path we are getting so it does

doing the copy don't take that colon in the selection so here here in this case we're just

pasting the required java home path here so you can find here so actually uh this is a put the

Java home actually I should write this one as Java home at hadup hyphen env dot

sh so this particular shell program has to be updated with the Java home path

so we have done that one and we have got the Java home path by just writing

the Java home dollar Java home we have written that one so getting the path

and we have done the pasting there now run the final command to set the set

up the hadup okay so we'll be going for this name note format will be going for the name note format so let us go

for that section now so yes we have done this now we shall go for this name note format so

copying and pasting it here so name note format is taking place yes it has got pasted

so formatting has been done so let me write these lines how to start the Hadoop so we're

supposed to issue it should be written in capital letters so dollar

hadup underscore installs slash sbin slash start hyphen all dot sH so this is the there's a

shell program you are supposed to execute you can find that Hadoop is now is getting initiated I should go for yes

so we're going to see what are the processes are running so here you can find

that node manager data node name node secondary name node we're having the

resource manager so these these are the respective processes are running in the

background so that means my hardube is on now let me open this one in the browser

so the command is local host colon 5 0070 so if you just issue this one in the

browser you can find that this particular hadoop has got so respective

GUI view we're getting here so in this way we have shown you that how to install

this hardup here so in the browser we are going to write local host colon 50070

to get the hardup overview page on so throughout this demonstration we have

shown you that how to install hardup on our system thanks for watching this video