Load Data from File into Pig Tables

In this video we are discussing load data from files into pick tables

So what is the respective comment? So load data from disk into pick tables

So to load data from HDFS file we need to use the load command

So the command will be load. It will take the data and split it by delimiter and stored as a given schema

So depending upon the given schema, the data will be stored. considering the delimiter for the splitting purpose

The load syntax will be this. So, the command syntax will be like this. So, variable name is equal to load input path, then using function as schema

So, this is the syntax for the load command. The function area, we can choose among peak storage

You can go for bin storage method. We can go for JSON loader, text loader, and in this way

And to see the table data, we need to use the dump operator

So, dump operator will display the current content of the table. So note will be there that we should start our Hadoop at first before issuing all these commands

So, I think for the better understanding, let us go for one practical demonstration to clear this concept that how these commands can be executed in the PIG environment

At first, we shall start Hadoop. so control alter t to initiate one new terminal here so to start hadup we should type dollar

hadup underscore install all in capital letters slash sbin slash start hyphen all dot sch so this is a

shell program which are which is supposed to get executed to start hadoop on our system

Now to check whether Hadoop is executing properly or not, we are executing the command

GPS and you can find that node manager, name node, resource manager, secondary name

node and data node all the processes are executing. Now we shall initiate our GUI interface of Hadoop So we are opening our browser In the URL we are typing local host local host colon 5 0070 slash you see the name node related information

is coming here. All the name node related information they are coming. So

Hadoop is operating fine. You can go for utilities, you can go for browse the file system and all the default folders and the files are coming here so

that indicates that my Hadoop is executing perfectly it has got loaded and initiated into the memory

now we shall show you how to run PIG on our system so we are going to open one terminal

so control altered T there is a shortcut and to open a PIG to use the local data in that case

the command will be PIG minus X local that means it can access the local data of the Ubuntu Linux but if I want to

access if I want to access the Hadoop data from the SDFS I shall issue only PIG

here so you see at the end we have got the grand shell here so now we shall

go for so this is a grand shell so at first we're going to have we're having

one data file and from the data file we shall load our data

onto a peak table so let us open another terminal here so here we are having one

data file which is there like this one so under this this G-y of this Sdf sdf s we

are going for this Hadoop my files from the root root of this Hadoop we are having

this Hadoop my files under this we're having student underscore info c sv so let us

see that what is the current content of it what is the current content so we shall go for SdFS

s df s dfs minus cat so that is the Hadoop route then Hadoop my files and then

student underscore info dot c sv you can find that the current content of the

file is this one that is where we have here we having five rows are there Amit Electronics Kolkata Dinesh Computer Science Chennai in this way we having five rows And this is the current content We shall fetch this content onto the pick table

So how to do that one? I'm just closing this terminal. So this is about grand shell

So we shall open, we shall create one peak table, that is the table, say, student

So student is equal to load. I can write this load. in capital letters also we can go for this haddub so what is the path here so

we shall go for this hardu my files m and f m and f these two letters will be

in the capital then the file name is student info the file name is student info

c sv so the file name with the path has been completed so I shall close

this single code then we shall go for this using pig storage p

capital and storage as capital so pick storage so I should have this one as

comma should be enclosed within single quotes and then we shall go for as name

the first will have the name name there is a card array then you shall go for

the second one there is a major that will be of the type of car

array then you shall go for this city that will be of the type of car array so I've

given three field names name major and the city so I'm going for enter so

student is equal to student is the big table going to get created is equal to

load and then we're having this had do my file let me check once whether it is

file or files yes it is Hadoop my files that is so let us go for this Hadoop my files

then we are having slash we're having slash student underscore info c sv that is the name of

the file should be kept within the single quotes using pick storage p and s capital

and then within bracket comma has to be enclosed within single quotes as name

card array comma major that is a card array and the city there is our car array so I pressing enter here so I think it has got created so to see the table description you can go for describe we can go for describe

then table name that is a student pick table name so you can have this name

of the type of card array major of the type of card array city of the type of

card array we can also write this describe in capital letters that will also work

Okay, now I want to see what is the current content of the pick table student so I shall go for this dump and student

Dumb student and then I shall give the semicolon so in the ground shell we're running all this queries in the grand sale

Yes, we're having all this data which has been fetched onto this where it has been fetched onto this

Pick table that is our student so it has got faced now we can limit the output if I want to see only the three tuples in the output then we

can use in this way I want to see only the three tuples in the output in that case

I shall go for say limit so I'm writing this one as LMT is equal to we can go

for this limit student three we can go for this limit student student is a

peak table name and three now if you go for this dump if you go for this dump

limit you can see the output that we're having only the omit dinesh and the

kussel tapples here omit dinesh and kusel so here also you can make this one say

I'm changing this one if you write this query in the command in this capital

later then also it will work so if I take this one as four if I take this one

as four so limit student four now if I go for this

dump LMLMT there is a limit you can find that we're having getting only the four

tuples are getting printed out of five so in this way we have shown you that how to create

a pick table and how to see the description of the table using the described comment

and how to see the content of the table using the dump comment and how to see partial

portion of the table using the limit comment thanks for watching this video