Basic HDFS Commands | Hadoop | HDFS

In this video we are discussing basic HDFS commands

So there are multiple different HDFS commands, so we'll be discussing some of them

So some basic HDFS comments are there. So HDFS commands are very similar with the Unix commands and some syntax and output

format may differ for the comments. So here we're having some set up comment list, LS

The description is that list all files with permissions and other details

MKDIR for the creation of a directory. RM, remove a file or a directory

Put, store file or folder from local hard disks to the HDFS

So, that is a put from the local harddicks to the HDFS

CAT, concatenate text or display file contents. Next one is the gate which is the opposite of put

Store file or folder from HDFS to the local disk. local disk. We are having this one as count. Count number of directory, number of files

and the file size. So, these are the set of commands and the respective descriptions were provided

and these commands will be going for further more details. So, some basic HDFS commands

so at first we are going for more detailing about the LS command. So, the syntax is HDFS

DFS minus LS, and then you can mention the path. So, this LFS. So this LAS, you are going to

L. S command will show all the files and folders which are given in the respective path

And it also displays the file permissions, the owner and group and modification time

file size, everything will be displayed. So, this is a command that is the L.S

How to issue this one, SDFS, DFS minus LAS path. Next we are going for the MKDIR command

So syntax will be HDFS DFS minus MKDIR path or the. directory name. Then we are having this the mkDII command is used to create a new directory inside

the given location. So, the new directory name will be given here and the path will be there

under this particular path the new directory will be created. Obviously, to run these commands

start the Hadoop first to run the SDFS commands. Next, we are having this RM command. So

RM stands for remove. So the syntax is. sdf s dfs minus rm and then the file path. So syntax is sdfs, dfs minus rm. Another option might be used

that is a minus r and that is a directory name. So the RM command is used to remove a file

or a directory. If the directory has some elements, we need to use RM minus R. So in the case

it will go for recursive deletion. So option to recursively delete all the internal elements of that

folder or directory. Next one is the put command. So syntax will be HDFS, DFS minus

put, path one, path two, and then destination. So put comment is used to store data

into SDFS from the local disk. It can take multiple arguments and all are source except

the last one. You can have multiple source arguments, but the last one will be denoting the

destination. That is the destination path in the SDF. So, put means it is just transferring files from the local system, local file system to the HDFS

We're going for the next one, that is a gate. So, HDFS, DFS minus gate, path two, and then destination

Get command is used to store data from HDFS to the local disk

It can take multiple arguments, all the source locations in SDFS, except the last one, and

The last will be the destination path in the local HUDDix. So here this particular using this get command, you can transfer files from the HDFS to the

local file system. Next one is a count. Count is another command is there

So syntax will be HDFS, DFS minus count and path. So count comment is used to count the number of directories, files inside the given directory

whatever the path you are given. So where the number of count of the files and directories will be provided

and it also shows the file size of the directory. So, in this way, you have discussed so many commands of SDFS

So let us go for one practical demonstration for your easy and better understanding

So here is the demonstration for you. In this video, we shall discuss multiple different Hadoop HDFS comments

So at first, we're opening one terminal. So Control Alter T and we shall open 1G edit

We're going for the commands. okay so I'm just opening this one so here we'll be typing all the list of

commands whatever you are going to initiate so we shall show you the command like

LS then mkDIR then we shall go for say we shall go for a put copy from local

then we shall go for get copy to local so these are the commands will be showing you

then we shall go for say count then we shall go for rm and then say cat and also our move mv so

these are the commands very common commands we are going to show you one by one so we will be

showing you the l s mkdiir put copy from local we shall go for get then we shall go for copy to local

so remember here in this case t and l is capital so copy is all in lowercase letters

andый and L will be in capital so we shall go for this RM and then CAT and then move okay so

this is the list of commands so 10 commands we are finding so we'll be going to issue them so I'm

just opening another terminal here I want to put them side by side so these two

terminals are there so now let me introduce my my one folder so we are going to

for this home under the home we are having one folder that is a haduk my files so you see it

is in the ubunto linux folder it is not in sdfs so here we are having this haduk my files

having two files are there one is a sample file dot txt and another one is the student

underscore info c sb we'll be working on them we'll be putting these files onto the sdfs

folders and then from there we'll be getting them back and we'll be working on them

So what is the current content of this sample file.txta and student underscore info

So let me show you that one also. So we shall go for we shall so we are under this

So we are going for this cat. There is a home slash Hadoop

My files. So let us go for LS at first so that I can see that what are the files are there

So here we are having this. So these are the list of files. so minus l i shall go for this one as minus l so we're having sample underscore file dot txt

and student underscore info c sb so let me check their contents so i shall go for this cat

then home slash we shall go for haddup my files slash sample file dot tx t

so CAD Hadoop my files sample files dot txc so these are content so going for the other

one so that is our student underscore underscore info dot CSB so these two file contents are in front of us so one is the sample underscore file dot txt another one is the sample underscore file Another one is the student underscore info

So I'm going for this browser here. So in this browser I'm going for the respective folder

You can find that under the HDFS route we're having no such folder called Hadoop My files

So I shall create one folder in the SDFS route. So this is a root you can find that

we'll be creating one folder under it so how to create it so you know that we went

for to open this one we went for the local host colon 50070 and then slash we got it and

from utilities browse the file system from here we came here so i want to create one folder here

the name of the folder will be hard to my files so i'm just going to create that folder so for that

I shall be using the command that is the mkDIR I'll be using the command that is mkdiar so how to use that one see we can issue s dFS minus mkdiir so there is a command and then it will get created under the root of the sdfs

haduk my files I think the folder has got created let me go for

a check so it is not being shown I can go for refresh you can find that

Hadoop my files this folder has got created you see the folder is empty you see

the folder is empty so their folder has got created so I can create such

folders using another way that is the Hadoop there is a Hadoop FS and rest part

will remain the same so as already we have created this is Hadoop My File, so I'm going for another one

I'm going for say Hadoop My Files 1. So this is the folder name

So I'm giving here. So Hadoop FS minus MKTIR slash Hadoop My Files 1

So if I go on entering, so if you come to this, if I go for refresh

you can file that another. folder has got created so that is very simple so either you can use HDFS DFS or

you can use Hadoop F s so how to remove this folder how to remove this

Hadoop my files one right now I don't require it so s DFS DFS DFS minus

we can also go for RM okay RM RM RM minus R means it will go on recursive all

the folders or subfolders or files whatever whatever deciding in this had to my files 1 will get deleted so there is a recursive

minus r so we shall go for this haddub my files 1 so sdf s dfs minus r m minus r

is recursive had to my files 1 so if I press enter the folder has got deleted

so you see the I shall go for the refresh you see the had to my files 1

it has got deleted so how to see this listing also from this from this

terminal so we shall go for say is DFS then DFS then minus LS slash so slash

slash means the hard-to-root we can find so these are the folders are there

how do my files are there these are the hard of my files we're having and this is

mine so if I go for this one so these are how do my files you are getting but

the Hadoop My files 1 it has got it has got deleted removed already okay so we

have done up to this so we have gone through this mkdir we have shown you that

how to do this mk d ir and also we have shown you that how to use this RM okay

also have shown you that how to use this LS this LS can also be done in this way

that is a Hadoop Hadoop FAADU fs minus ls root so also we can use the same okay so this much we have done going for clear

now i want to copy those two files for which are the files so ls minus l so that is our root

there is a home folder hadupe my files I have to see all the files there so we're

having sample underscore file dot txt and student underscore info info c sb so already

i have created this folder here so i want to copy them to this particular folder so that means from the ubunto Linux file system to the sdfs i want to copy

so in that case what we can do we can issue the command is dFS then dfs then

minus put we can go for minus put okay so from where from the home from the home

we are going for home and then had to my files from the home we shall go forward

had to my file slash what is a file name that is a sample file

so under which folder under root haddub my files so under this folder I want to copy this respective file so that is the sdFS dFS minus put

we're going for the um Ubuntu Linux file system that is a local file system so root root

will be denoted by the tilt sign had to my files is the respective folder name sample

underscore file dot CSV uh is a sample underscore file dot txt i think it is the txd one so I shall go

for this one as tx t okay so there is a tx t and then it will be a copy to this folder

that is the root of the sdf s and then had to my files okay so let me check

whether it has been copied there or not so I shall go for this hadup fs minus

ls then I shall go for this one as hadup my files

you see the folder that is a file that is a sample underscore file dot txt has been copied

there okay so the command is put so instead of put i can also use copy from local so i'm just

erasing this put here i'm going for copy from local i shall go for this copy from local i'm sticking

with the same command but only the file name i'm going to change so here i shall go for this one as

student info c sv student info dot c sv you can easily find so one file we're

having there is a student info dot c sv so this file I'm going to copy here so

sdf s dfs minus copy from local so there is a root folder so it is a home

folder rather then had to my files student so there is a student

underscore info c sv to the respective sdfs folder that is the root had to my files so if

i go for this so copy from local is working so let me go for the ls i can find two files

have got copied there so one i copied using the command put another one i copied using the command

copy from local so put i have shown you also have shown you this copy from local here okay now let me

go for here you can see that if I click I finding both the files here okay now I want to make a copy of this say sample underscore file dot tx t a sample underscore file 1 so how to do that one

So let me come down to this here. So I shall go for this copy. Okay, see I shall go for

uh, hadu, s dFS, dFS minus CP. SdFS dFS minus CP. So root, haddued, hadoo

my files then I'm having say sample file.csb sample file dot CSV in the same

folder I'm going to copy others I can select other path also as we usually do

had do my files slash sample underscore file 1 dot CSV

So sample underscore file dot tx t rather okay so I shall go for this one is tx t so see the

command that is the sdf s dfs minus cp hadu my files sample underscore file dot tx t

to this hadu my files sample underscore file 1.t xt so here I'm going to show you the

command that is a cp here we're going to show you the command that is a cp so let me

press enter okay so let me go here you can find that go for this one you

can find that another file has got created there is a sample file 1 dot tx t okay so

now I want to rename this one student info dot says I want to rename this one

sample file 1 dot tx t to sample file 2 dot txt how do that one okay so let me

come back so another comment we have discussed here there is a CP

so we have covered that one so let me go for the renaming okay let me go for

the renaming so here what what we are happening so let me go for this

LS minus else this one so let me go for clear that will make the screen clear

yes now so LS minus L slash slash Hadoop my files how do my files

so these are the files I'll be working with so sample files

sample underscore file dot tx t it has got rename to sample underscore file 1 dot txt so I

want to I want to rename it okay see so I shall go for sdfs DFS minus move

so what is the path root hadu my files so I shall go for sample

underscore file 1. tx t to had to my files had to my files slash sample underscore files sample

underscore files let me check one what is file 1 dot t x i think let me check one's please so

there is a file 1.t x yes so now i shall come back so there is a sample files

2. dxt so that is a move we are going to do coming down here now it has got

renamed to files 2. txd so while doing this moving also we can make the file

move to the different path so let us go for say Hadoop Hadoop F.S minus mk DIR so under this

Hadoop my files I shall go for creation one the directory that is a

say directory one I'm making a directory that is a directory one here okay I

think I've done it let me check once so I shall come back here yes the

directory one which is empty right now which is empty so let me move that file

so I shall go for this move встретoop there is a file let me go for this so I shall make this one as file 2 dot txt

so this particular file I want to move to this directory one I want to move this

directory one okay let me check once so now if I go for this refresh if I go

for this refresh so file has not been copy it so let me check once again so I'm just making the file name as

sample underscore file 2 dot txd the sample underscore file 2 dot txt let me check

whether the file is existing there or not so that is a sample

underscore files 2. txt yes it is existing so here the file name should be

files 2 so that was a mistake so I I'm making this one as files 2

So that is a HDFS, DFS minus move. Haddub my files sample files 2. dxt

Haddub my files, DIR 1, I'm making this one say file 2

D x t. You can find that also I'm moving it to a different path

and also renaming it at the same time. So I'm going for this

So let me check once. So I'm going for this DIR 1. Yes, the file 2

D x.d. So in this way you can find that how this move command can be issued here

So move I have shown you. Next I shall go for the content

I want to show you the content here. So how to show you the content

So I shall go for this one as say Hadoop, fS, then minus GAT, then slash Hadoop, my files

So here we are having student info. You can find that we are having this student info.csv

You see the current content is being shown. Going for clear. So let me show you again

Using the same command, but here I shall go for HDFS. DFS

You can find that I'm getting the same content. that is our cat so I've shown you how to use this cat so this much we have done now

what I shall do I shall go for say for the removal of the directory also will be

using this RM I shall show that one next I shall go for this get now from the

SDFS file system I'll be copying our file to our local file system so for that

we'll be using the command get okay so let me check what are the files we're

having here okay now see so I shall go for this SDFS dFS minus LS so we're

having this one as hadu my files okay so these are the files we're the files we're

having there is a sample underscore file.txt and student underscore info c sb so for both the files are existing I want to copy them so to my local

folder so I'm just making this one in this way so before going there I'm just

showing you that what are what are the files I'm having here so I shall go for

this home then had do my files you can find that we are having this okay so So this is the file which is in the SDFS file system I want to copy it to this Ubuntu Linux That is a local file system with the renaming

So how to do? How to do? So we shall go for HDFS. DFS minus get slash

There is a Hadoop root. So Hadoop My Files. So what is the file name

there is a sample underscore file dot txt to where so I shall go for this go for

this one there's a home Hadoop my files I'm making this one as sample 1 dot tx t

okay so I'm just getting the Hadoop file system one file from the Hadoop

had to the file system to the local file system so see the command so that is our

is dFS dFS minus get slash hadu giles slash sample underscore file increasing to

the home directory hadug my files sample 1 dot txt so if I do the copy here so

let me see the current content you can find that sample 1 dot tx t has been

copied here okay now I shall go for the and another time I shall issue the same command but here I'll be doing some changes so

here I'll be copying the file that is a student info so I'm going for this student info

so student one dot CSV will be the file name okay so here I shall go for

Hadoop Fs so I want to show that also it will work so instead of get I shall use

use copy to local so I'm I'm showing you in all possible combinations so

Hadoop fs minus copy to local so slash Hadoop my files so what is the name

here so that is a student info dot CSB so I'm just making this one okay so

so here I shall go for this student info c sb so that is a hadoop fs

fs minus copy to local slash hudub my that is a sdfs root so root that is a

slash hadoop my files student info dot csv to the local folder so that is a home

then had to my files and then student one dot CSV so I'm just copying it so if I go

for the check you can find that that student one dot CSV the file is existing

here so in this way I have brought both the sample one dot txd student one dot

c sb from the had to the local file system so I have demonstrated the use of

this get and also the use of this copy to local So now let me erase the all the full content let me erase this content of this had to my file so I do not want to have such content here and to delete this folder

So you see this particular folder is going to get deleted so how to do that one so in that case I shall go for

So let me type type here so SDFS DFS minus RM minus R that is a recursive root there is a sdFS root so we shall go for this

Hadoop my files so you can see the all the all the files all the subfolders whichever

is existing within this Hadoop my files folder will get deleted along with the

Hadoop my files folder will get deleted itself so I shall go for this enter so now

let me check whether it is existing there or not so I'm going for the refresh you

can find that that Hadoop my files that folder has got deleted with its all

content so in this particular demonstration we have shown you that how to use the

commands like LS mkdir put copy from local get copy to local then RM CAT mv and

CP so we are remaining with the count okay we are remaining with the count so

how to go for the count we can go in this way you can go for say say I shall go

for say SDFS s dFS minus count now we can give the

route also you can see that how many how many folders are there how many files

are there you are getting the count along with the byte count you are getting okay so these are the different folder so let me make one folder so s dFS

s dFS so my dFS say minus mk d i'm making the folder again so i shall go

for say slash had do my files i'm making this folder at first

okay to this folder I'm copying two files as I did earlier so s DFS then I shall go

for this minus put I shall go for this minus put from where from our local system

home then haduk my files slash what is the name that is a sample file dot tx t

to had to my files so i'm just making a copy of this okay then i'm making another

file there so that is my student info whatever i did earlier i'm just repeating one second

so that you can also see csv

so let me go for the LS so hadduf fs minus LS slash haddup my files so we're

having this sample underscore file dot tx t so this particular student I think the

spelling was wrong so that's why so student info and c sb yes i think now it has got copied so let me go for the file check yes the two files are

there one is the sample underscore file dot txt and the one is the student underscore info

c sb so let me check it from here also so the had do my files is not there if i go yes

it is there and it is having two two files are there okay now let us issue the command that is

the count s dFS minus dFS dFS then i shall go for minus count haddub my files so it is showing that

it is having 294 bytes of information and here we're having two files are there

and there is one folder so in this way it is showing you the respective count

so there is a respective count is there so these are the two files are existing under

this my Hadoop my files folder so in this way I have demonstrated and the all the

commands that is the LS mkDIR put copy from local and then get copy to local then

count RM CAT MV and CP I think now the conception is getting clear to you how

to issue such commands in our Hadoop HDFS. Thanks for watching this video