videocam_off
This livestream is currently offline
Check back later when the stream goes live
Up next in 10
Spark Action Functions
Show More Show Less View Video Transcript
0:00
in this video we are discussing spark
0:02
action functions in the previous video
0:05
we have discussed spark transformation
0:08
functions we know that all the
0:09
operations on RDD can be divided into
0:12
transformations and actions here we are
0:15
considering action related functions we
0:18
know that in case of action related
0:20
functions it will take our duty as input
0:22
and it will produce non or Duty result
0:25
as output and that is a main difference
0:27
between transformation and action so let
0:30
us discuss some of the very important
0:32
and frequently used functions in action
0:37
so now some of the action functions are
0:40
here we are having this function and the
0:42
respective description so reduce is a
0:45
one function and aggregate the elements
0:48
of data set using a given function it
0:51
will go on doing the aggravation of our
0:53
data set and then we are having this
0:55
collect get all elements of the data set
0:58
as an array so all the elements of the
1:01
data set will be converted to as an
1:03
array and that array will be the output
1:05
next one is a count function returns
1:09
number of elements in the data set so it
1:12
counts the number of elements in the
1:13
data set next one is the first function
1:16
take the first element of the data set
1:19
so it returns the first element of the
1:21
data set next one is a take in n is the
1:24
input argument so take the first in n
1:27
elements of the data set so it will take
1:29
or select only the first n number of
1:32
elements of the data set next one is a
1:35
tick ordered in so it is only take in
1:38
but it is take ordered in so take first
1:42
in elements by the natural order or by
1:45
using the comparators so using
1:48
comparators it will be taking the first
1:50
n elements of the data set next one is a
1:55
count by he so it is available when our
1:59
duty is like K comma V so K stands for
2:03
ki and V stands for value so it is
2:05
available when our D D is like our K
2:08
comma V that means the it is in the form
2:10
of he value pair and it
2:12
turns the hashmap of k comma int with
2:16
the count of each keys so how many times
2:19
the key has got counted so it will go
2:21
under counting on that so k comma int
2:24
will be the outcome and that is a count
2:26
by key function next one is a for each
2:30
function returns a function for each
2:33
element of the data set so it will
2:36
return the return a function for each
2:38
element of the data set so that is the
2:40
purpose of this for each function save
2:44
as text file so that is another function
2:46
write the elements of RDD into our text
2:50
files so elements of our didi will be
2:52
written on tune on our duty output that
2:54
is our text file save as object files so
2:58
that is another function write the
3:00
elements of our didi in a simple format
3:03
using Java serialization so here you are
3:06
having the function name and the
3:08
respective descriptions so here in this
3:11
particular video we have discussed
3:12
multiple different functions in actions
3:15
thanks for watching this video
#Programming