Installing Hadoop on Pseudo Distributed Mode

In this video we are discussing installation of Hadoop on pseudo distributed mode

So how to install this one? Let us go step by step

Installing Hadoop in pseudo distributed mode, to set up Hadoop for pseudo distributed mode, set up the following XML files located in this folder

You see, this path is there and there is a folder under this particular folder where having these three files are there

Multiple XML files are there but we are concentrating only on these three files and each and every file where we'll be putting some set of XML

So, first one is the core hyphen site. XML, SDFS hyphen site

XML and map rate hyphen site dot XML. So three XML files are there and they require some update

So set up core hyphen site. That was the first in our list

we are going to update that one. So, open core hyphen site.xml from the given path

You see, we had this part and this is the respective file name here and put the following

lines in the XML files. So, you can open this file. It is a text file. You can open it

using G-Edit and then put this files, put this lines onto the respective file

Next was HDFS-S-Sight.coml, which will be found in this respective folder. And

This is the code, XML codes, are to be written into the file

So, open the file using G-Edit or any other text editor and put this XML code there

And the last one, there is a MapRed-Sight. It will be found in this respective folder

And if the file is not present, copy the MapRet site. Dot XML dot template

So, this is a file will be there. So, there is a map rate hyphen site.xml

Dot template file and rename the copied file with the map rate hyphen site

So, if the file is not directly there, you will be having one template file

So, just make a copy of that and rename it to this name and put the following lines which

are written in XML to that file itself. Open the hadoof hyphen env

v.s.h from the given path. So, this is the path from where you are supposed to open this

shell program. The shell program is there and export the Java underscore home environment

variable in that file. So, how to do that one? Put the following lines there. There is

the export Java-h-home. There is a underscore here. Export Java-unterscore home is

equal to slash user slash local slash jdk 1.8.0 underscore 171. So, just see your path

In my system, the path was like this, whatever I followed the steps. My path was this

But if you're having some different version of Java, then obviously the respective export

the string will be also changed. So, export Java underscore home is equal to. So this line has

to be put in the respective file. There is a Hadoop. hyphen env.sh. So I think it is better. Let us go for one demonstration to show you that how the

steps are to be carried out and what are the different ways to do that one for your better understanding

So here is the demonstration for you. So in this video, we are going to discuss how to install

Hadoop version 2.4.1. So before installing Hadoop version 2.4.1, we are going to install SSA and

R-Sync. So what is SSH? S-S-H is nothing but secure shell is a secure shell and it is a

cryptographic network protocol for operating network services over an unsecured network and that is

the purpose of S-S-H. Now what is R-Sync? So R-Sink is nothing but is a utility for efficiently

transferring and synchronizing files across computer systems by checking the timestamp and the size of the file

And this R-Sync is having the full form, that is a remote sync

And here also we're going for the setup for SSH without pass phrase

Now actually this pass phrase is actually this concept is required for the additional security here So to install SSH the command is pseudo APT install SSH

To install R-sync, the command is pseudo-a-pT-hyphen get-install R-sync. So we'll be issuing these commands in the terminal prompt

And set up for SSH without passphrase is the command will be SSH-H-K-gen

hyphen T, RSA. while installing this we'll be pressing enter for three times to ignore the next

stages so let us go for the demonstration that how we are doing going to do this

one so at first we are going to install this hardu version 2.4.1 highlighting this

one and now we are going for the first command that is a installation of SSH going to

the terminal clearing it up so now here we're issuing the command pressing

enter the command is getting executed yes installation has been done yes the installation is still in process and it has been

done successfully now we are going to copy this pseudo APT hyphen get install

R sync so this is a command we are going to issue now so we are doing the

copy and then coming to this paste and the command prompt has been

obtained now we upsetting we are going for SSH without pass phrase so copying this

command here and you shall press three times enter you can find that and here

we are pressing enter three times you can find that we have entered three times

and here we have got we have got back the prompt so that means the instruction

has been done successfully now here we are copying this particular file and

actually we are concatenating it at the end of the file that is the slash dot ssh slash authorized underscore keys so the left and side file will be copied at the

end of this right-and-side file if the file on the right-hand side does not exist then the file will be

created and the new content will be pasted there so this is a command we are going to issue now so

let us go for this copy and issue this command so comment has been issued you can find that now

we shall come back we shall issue the command that is our s-sh local host so we're going to

for this marking SSH local host we shall go for yes pressing enter so the

prompt has been obtained now here we're having some steps are there so we'll

be downloading this Hadoop version 2.4.1 from the link as it has been given it

we shall do the copy and paste onto the URL box so let us go for that

operation and the Hadoop will get installed on our system copying this URL then

opening the browser and go for paste and go now you are going to select the 132 mb

file that is the Hadoop 2.4.1.tddddd going for save the file is getting downloaded

on our download folder you can click here you can see the progress how the

file is getting downloaded here so now it is almost getting completed the download has been done onto the

download folder opening the folder now so we are creating one folder

under home the name of the folder will be hadup giving the name and now all

the extracted files from the zip file we've done the dragging it so that all the

files will get copied onto our hadu blank folder shall close this one so this is our Hadoop folder so we have followed the step

that is the extract the downloaded files into the Hadoop folder so this much

we have done now we are going to edit about a file that is a slash ETC slash

profile file and the following two lines are going to be added at the end of the

file that is a profile file which is residing under the ETC folder so let us go

for opening this there is a ETC slash ETC slash profile file so let us go for opening so pseudo g edit going for slash it is

slash profile opening the file it is asking for the password giving the password here cannot open I think we should terminate this terminal let us go for another one let us open another terminal here okay so another terminal has got open

we're going for this pseudo let me execute this one again so pseudo g edit slash

etc slash profile pressing enter giving the password so file has got opened we're

coming at the end creating some space here now coming to this install and copying

these two lines so here we are exporting this hardup underscore install this variable is

getting initialized with this path and also export path so is equal to dollar

path colon ha dollar had doop underscore install slash bin so these two these two

lines were just doing the copy and paste at the end of

ETC slash profile file so doing the paste so file has got updated now saving it then I

shall close save and close then I shall execute the ETC profile file so that

the respective changes will get effective in the system okay we have done it so

execute the slash it isse slash profile file so this much you have done now hadup pseudo

distributed mode so we'll be going for some updates in the xml files and so on so let me go show you

so at first we are dealing with this core hyphen site dot XML so this is a file where we are going

to do some updates and this file is residing under the folder that is a hadup slash etc

slash Hadoop slash core hyphen site.xml so this is a file is there where we'll be

pasting this configuration portion the portion which is written in this configuration tags so let me open this core hyphen site

So it is under this Hadoop it is C hadoop and here we're having this core

C yes C is the starting character so I shall be going for core hyphen site.xm. So opening this one here you can find that

this configuration tags are blank. So let me copy this one at first. So I'm done the copy and then

I shall go for this core hyphen site.t.xml we have done the paste. You can find that only we have

just updated the portion. We're not touching any other portion of this file, just updating that

portion whatever you wrote. You can pause the video. We can do the

same you can make this XML ready and then we are saving and closing it so we have done

this code hyphen site dot XML so required update we have done now we are going for

the next one there is a Hadoop slash ETC slash Hadoop slash HDFS hyphen site dot XML so

that is another XML file is there just remember this path here so Hadoop slash ETC slash Hadoop

slash sdFS hyphen site.xm. So these are XML file. We're going to do some updates

We are going to carry out some updates on this particular file here. So what is update

So here this configuration portion has to be written there. So let me open the

open the file now. So doing the copy here. So sdFS the file is already open. So that is

the sdFS hyphen site dot XML. So that is the sdFS hyphen site dot XML. So

here you have done the copy and paste here so configuration tags have got

updated so here we are having the replication factor of one we can also go for

three so here we have gone for the replication factor dfs dot replication so this

particular property has been set with the value one here okay so we have done

done it now you shall go for the save and then we shall go for the close

next we're going for the next file which is the map red

which is residing under the folder that is the Hadoop slash etec slash Hadoop

map red hyphen site so this is a required file where this configuration section has to be updated So here you are putting this map rate dot job tracker and local host colon 9001 so name and the value so they have they have got enclosed

with the respective tags. So we're going for this so here we can find that we are not finding any map read

hyphen site dot XML we're not finding this one so what you are finding we're

getting map red hyphen site dot xml. dot template so what we shall do we shall make a copy and paste so we shall make a replica

of this template file and then we shall change the name to map rate hyphen site

xml and there will be doing the required updates in the tags so let me make a copy

so I'm done the copy and paste so it has made a paste here so now I shall go for

this template and this blank space is to be eliminated yes we have made the thing

so now coming to this installation details. tx taking this configuration portion okay now going for this map redside.xml this

configuration portion is getting updated with a new content you can find that

here we have done the copy and paste so please remember all these steps

because all these steps are necessary and they are to be done the order in

which you are demonstrating here so now we shall go for this save and exit

okay now we're going for this Hadoof EnV Hadov hyphen env.sh. So here we are supposed to put the Java path here, Java home path. So this is a Java

home path we are getting. So it just doing the copy. Don't take that colon in the selection

So here, here in this case, we're just pasting the required Java home path here. So you can

find here. So actually, this is a put the Java home. Actually I should write this one as Java

home at hadup hymnv.com.env. dot s h so this particular shell program has to be updated with the java home path so we have done that one and we have

got the java home by just writing the java home dollar java home we have written that one so getting the path and we have

done the pasting there now run the final command to set the set up the hadoop okay so we'll be going for this

name note format will be going for the name note format so let's

let us go for that section now so yes we have done this now we shall go for

this name not format so copying and pasting it here so name not format is

taking place yes it has got pasted so formatting has been done so let me

write these lines how to start the Hadoop so I'm supposed to issue it should be written in capital letters so dollar hadup underscore

install slash sbin slash start hyphen all dot sH so these are this is a this

is a shell program you are supposed to execute you can find that

hardook is now is getting initiated I should go for yes so we're going to see what are the processes are running so here you can find that

node manager data node name node secondary name node we're having the resource

manager so these these are the respective processes are running in the background so

that means my hub is on Now let me open this one in the browser

So the command is local host colon 5-070. So if you just issue this one in the browser, you can find that

This particular Hadoop has got so respective G-Y view we're getting here

So in this way we have shown you that how to install this Hadoop here. So in the browser we are going to write local host colon 50070 slash

to get the Hadoop overview page on. Throughout this demonstration, we have shown you that how to install Hadoop on our system

Thanks for watching this video