Hadoop Cluster Architecture

In this video we are discussing Hadoop cluster architecture

Hardup cluster architecture is consisting of name nodes, also known as primary name node

and data nodes and secondary name node. Name node is actually containing the metadata, that is the data about the data regarding the data nodes

That means it will contain the file system information, where the data will be stored and data has been stored

So that information, replica information, all will be kept in the data

the primary name node. In case of data nodes, so they are actually responsible for storing

data. And secondary name node is not there to replace the primary name node. So, secondary

name nodes are there to give the help or to give a support to the primary name node. So

let us go for more discussion on this topic. So what is Hadoop cluster architecture

Haddub clusters are set of commodity hardware. Commodity hardware means, hardware that means here the hardware cost is very low as a result of that data

storing cost will be also getting low they are used to store and process parts

of data it is basically the file system of the Hadoop and which is also known as

HDFS that is Hadoop distributed file system now let us consider this diagram

this is the name node and here the metadata related operations will be done and it is containing differing kinds of metadata that is the name replica information file system information

and so on and here we are having multiple data nodes divided into multiple racks so here

this data note is actually responsible to store the information and all the replica related

information will be also done here that means one data residing in one data note will

have a replica in another data note accordingly. This name node will be doing some block-related

operations on these data nodes and these data notes will be accessed by the client to perform

the read-write operations. At first, the client will be asking the name note to get the required

metadata. After getting the metadata, this client process will be accessing the data notes for the

data read and write information. These data notes are having the responsibility to send its

health report to the name node after regular time interval. And that is the basic architecture

There is a Hadoop cluster architecture. So let us discuss each and every type of nodes separately

So what is the tax of the name node? So we are discussing the name node at first

The name note is used to store the metadata and another data related to data nodes

So it will be actually storing the metadata. There is the data about data of the data

data nodes onto the name nodes and the name note is also responsible for managing the file system namespace it controls the access of different clients into the data blocks so which client

will be allowed to access a certain data block on the data node or not so that will be controlled

by this name node periodically checks the availability of the data nodes and also it also

cares about the replication factor of the data blocks so these are the different

responsibilities which will be handled by name note next we are going for what is the

task of data nodes data notes are the main storages of data

hardook uses low-cost hardware to store data in commodity hardwares and data

notes are responsible for storing replication creating deleting these types of jobs according to the instruction of the name node so which data has to be

replicated which has to be deleted so all this information which will be coming from the name

node to the data note data note is responsible to obey and execute those instructions these data

notes send the health report to the name node periodically and the default time is three seconds

so after every three seconds these sent the report to the name note so these data notes will

be sending the respective health report to the name node at each third second of time

Next we are going to discuss that is the secondary name note So whatever the name note we discussed earlier can also be called as name note or primary name note Secondary name note that is a secondary name note is another specially dedicated node

which is used to take the checkpoints of the file system. So this secondary name note is having the main responsibility to take care of the checkpoints

of the file system. The secondary name note is not for the substitution of the primary name note

name note and it helps the name note but not replacing that name note so it is actually giving

a support to the primary name node so let us come to this point here we're having the

secondary name note and there we're having the name node or the primary name note so this primary

name note is accessing this fs image that is a file system image so now this secondary name

node will do the update of this fs image with edit locks and this fs image will be updated

and its copy copy the update of the fs image back to the name node so that the name note can get the fs image perfectly so it is taking the care of the checkpoints of the file system so while doing this update before that it is querying for edit locks in regular intervals with the primary name note so after getting that information it will do the updates and updated copy will be copied here so as a result of that name node will get the always the updated fs image

So, in this way, in this discussion, we have discussed what is Hadoop cluster architecture

Thanks for watching this video