Up next in 10
Join this live session with Magnus Mårtensson ft. Karl Ots for the next episode of The Cloud Show with Magnus Mårtensson on November 27, at 01:05 PM (EST).
The show is about cloud leadership and all the important questions relating to cloud projects. Certainly, many matters when a company is going to and wants to be successful in the cloud, are about technology. However, there are many additional matters, adjacent to technology, that we also need to tend to regarding business strategy, human resources, organizational change, planning for a technical cloud approach, and many more questions. These conversations are critical for a healthy cloud and for a swift and accurate cloud approach.
GUEST SPEAKERS
Karl Ots, is a cloud security leader and author with over 15 years of experience in the technology industry. He is the Head of Cloud Security at EPAM Systems, a global engineering and consulting company, where he leads a team of experts in delivering secure and compliant cloud solutions for Fortune 500 clients across various industries.
📺 CSharp TV - Dev Streaming Destination http://csharp.tv
🌎 C# Corner - Community of Software and Data Developers https://www.c-sharpcorner.com
#CSharpTV #csharpcorner #TheCloudShow #CSharp #Interview
Show More Show Less View Video Transcript
0:01
hello Hello friends welcome to yet
0:04
hello Hello friends welcome to yet
0:04
hello Hello friends welcome to yet another episode of the cloud show and
0:06
another episode of the cloud show and
0:06
another episode of the cloud show and today we're going to be talking about
0:09
today we're going to be talking about
0:09
today we're going to be talking about generative AI because that's a Hot Topic
0:11
generative AI because that's a Hot Topic
0:11
generative AI because that's a Hot Topic why not and I have brought to the show
0:14
why not and I have brought to the show
0:14
why not and I have brought to the show today I'm very happy to have as the star
0:17
today I'm very happy to have as the star
0:17
today I'm very happy to have as the star of the cloud show a very good friend of
0:19
of the cloud show a very good friend of
0:19
of the cloud show a very good friend of mine his name is Carl Lots
0:22
mine his name is Carl Lots
0:22
mine his name is Carl Lots [Music]
0:32
well hello there
0:34
well hello there
0:34
well hello there Carl hey Magnus glad to be here very
0:37
Carl hey Magnus glad to be here very
0:37
Carl hey Magnus glad to be here very very welcome to the cloud
0:39
very welcome to the cloud
0:39
very welcome to the cloud show so happy to have you all right so
0:42
show so happy to have you all right so
0:42
show so happy to have you all right so we're gonna talk about generative AI
0:44
we're gonna talk about generative AI
0:44
we're gonna talk about generative AI because um you know we'll find out the
0:46
because um you know we'll find out the
0:46
because um you know we'll find out the reason for that that a little bit later
0:48
reason for that that a little bit later
0:49
reason for that that a little bit later but you um we should introduce you first
0:52
but you um we should introduce you first
0:52
but you um we should introduce you first to the audience briefly um tell us uh
0:54
to the audience briefly um tell us uh
0:54
to the audience briefly um tell us uh quickly who you are U and and like what
0:57
quickly who you are U and and like what
0:57
quickly who you are U and and like what do you do yeah absolutely so uh so my
1:01
do you do yeah absolutely so uh so my
1:01
do you do yeah absolutely so uh so my name is Carl I'm a security uh MVP after
1:05
name is Carl I'm a security uh MVP after
1:05
name is Carl I'm a security uh MVP after a few years being an Azure MVP where I
1:08
a few years being an Azure MVP where I
1:08
a few years being an Azure MVP where I started from um and working working for
1:12
started from um and working working for
1:12
started from um and working working for a long time in the in the in the cloud
1:13
a long time in the in the in the cloud
1:13
a long time in the in the in the cloud just like just like you Magnus um know
1:16
just like just like you Magnus um know
1:16
just like just like you Magnus um know know each other for quite quite some
1:18
know each other for quite quite some
1:18
know each other for quite quite some time there on that scene also an RD um
1:21
time there on that scene also an RD um
1:21
time there on that scene also an RD um in my day job I'm heading Cloud security
1:23
in my day job I'm heading Cloud security
1:23
in my day job I'm heading Cloud security at Gan assistance which is a Global
1:26
at Gan assistance which is a Global
1:26
at Gan assistance which is a Global Engineering Company um large uh large
1:29
Engineering Company um large uh large
1:29
Engineering Company um large uh large 50,000 people people company um managing
1:32
50,000 people people company um managing
1:32
50,000 people people company um managing different different type of products and
1:33
different different type of products and
1:33
different different type of products and projects and yeah I'm I'm you know
1:36
projects and yeah I'm I'm you know
1:36
projects and yeah I'm I'm you know running running off the bits and pieces
1:38
running running off the bits and pieces
1:38
running running off the bits and pieces that comes with with running a security
1:40
that comes with with running a security
1:40
that comes with with running a security practice so business people products
1:42
practice so business people products
1:42
practice so business people products delivery all of the fun things of
1:44
delivery all of the fun things of
1:44
delivery all of the fun things of escalations uh what when projects don't
1:46
escalations uh what when projects don't
1:46
escalations uh what when projects don't go right and and also the fun things of
1:48
go right and and also the fun things of
1:48
go right and and also the fun things of uh when when we are building something
1:50
uh when when we are building something
1:50
uh when when we are building something new for some of the biggest clients out
1:53
new for some of the biggest clients out
1:53
new for some of the biggest clients out there that is that is quite a list and
1:56
there that is that is quite a list and
1:56
there that is that is quite a list and uh well since we know each other I know
1:58
uh well since we know each other I know
1:58
uh well since we know each other I know for a fact that you're you're awesome uh
2:00
for a fact that you're you're awesome uh
2:00
for a fact that you're you're awesome uh accent it's not very very evident but if
2:03
accent it's not very very evident but if
2:03
accent it's not very very evident but if you really know then you know that you
2:04
you really know then you know that you
2:04
you really know then you know that you have a Finnish accent because you are
2:06
have a Finnish accent because you are
2:06
have a Finnish accent because you are originally from Finland and then you
2:08
originally from Finland and then you
2:08
originally from Finland and then you lived in Switzerland and now you live
2:11
lived in Switzerland and now you live
2:11
lived in Switzerland and now you live in yeah uh so I've been moving moving
2:14
in yeah uh so I've been moving moving
2:14
in yeah uh so I've been moving moving around uh a few times my my accents is
2:18
around uh a few times my my accents is
2:18
around uh a few times my my accents is is is indeed from Finland but but yeah
2:20
is is indeed from Finland but but yeah
2:20
is is indeed from Finland but but yeah right now I'm based in in the US I'm
2:22
right now I'm based in in the US I'm
2:22
right now I'm based in in the US I'm based in in the east coast New York City
2:25
based in in the east coast New York City
2:25
based in in the east coast New York City close close to our
2:27
close close to our
2:27
close close to our headquarters yep so let's talk about
2:29
headquarters yep so let's talk about
2:29
headquarters yep so let's talk about generative Ai and specifically you well
2:32
generative Ai and specifically you well
2:33
generative Ai and specifically you well you don't have a book to show on that
2:34
you don't have a book to show on that
2:34
you don't have a book to show on that yet but you do have another book on
2:36
yet but you do have another book on
2:36
yet but you do have another book on security that you already wrote and and
2:39
security that you already wrote and and
2:39
security that you already wrote and and apparently you're a glutton for
2:40
apparently you're a glutton for
2:40
apparently you're a glutton for punishment because you are writing
2:42
punishment because you are writing
2:42
punishment because you are writing another book now on generative AI tell
2:44
another book now on generative AI tell
2:44
another book now on generative AI tell us about it yeah yeah that's right so so
2:47
us about it yeah yeah that's right so so
2:47
us about it yeah yeah that's right so so as my pandemic project more or less I I
2:50
as my pandemic project more or less I I
2:50
as my pandemic project more or less I I wrote a book on on Azure security and it
2:53
wrote a book on on Azure security and it
2:53
wrote a book on on Azure security and it was it was all all fun and games for for
2:56
was it was all all fun and games for for
2:56
was it was all all fun and games for for sure but uh I basically didn't I didn't
2:59
sure but uh I basically didn't I didn't
2:59
sure but uh I basically didn't I didn't kind of Scrat R service service in a lot
3:01
kind of Scrat R service service in a lot
3:01
kind of Scrat R service service in a lot of these areas I didn't go as deep as I
3:03
of these areas I didn't go as deep as I
3:03
of these areas I didn't go as deep as I could as was mostly focusing on kind of
3:05
could as was mostly focusing on kind of
3:05
could as was mostly focusing on kind of more architectural pie there um and
3:08
more architectural pie there um and
3:08
more architectural pie there um and after a few years uh between the the
3:10
after a few years uh between the the
3:10
after a few years uh between the the previous book and and this one uh geni
3:13
previous book and and this one uh geni
3:13
previous book and and this one uh geni came came in as a Hot Topic I I brought
3:16
came came in as a Hot Topic I I brought
3:16
came came in as a Hot Topic I I brought a few pieces uh did some pieces of
3:18
a few pieces uh did some pieces of
3:18
a few pieces uh did some pieces of content spoke a few times had a couple
3:20
content spoke a few times had a couple
3:20
content spoke a few times had a couple of projects uh from the beginning with
3:22
of projects uh from the beginning with
3:22
of projects uh from the beginning with with some clients and and I found that
3:25
with some clients and and I found that
3:25
with some clients and and I found that there is there's a lot that we still
3:27
there is there's a lot that we still
3:27
there is there's a lot that we still don't collectively of course know but
3:29
don't collectively of course know but
3:29
don't collectively of course know but there's still a lot that like I feel I
3:30
there's still a lot that like I feel I
3:30
there's still a lot that like I feel I could just share from uh from the
3:33
could just share from uh from the
3:33
could just share from uh from the already this couple of years of of
3:34
already this couple of years of of
3:34
already this couple of years of of experience and also adding that kind of
3:37
experience and also adding that kind of
3:38
experience and also adding that kind of cloud background how does how does
3:39
cloud background how does how does
3:39
cloud background how does how does someone who's uh who who knows to secure
3:41
someone who's uh who who knows to secure
3:42
someone who's uh who who knows to secure cloud look at the new technology
3:43
cloud look at the new technology
3:44
cloud look at the new technology specifically in this cases J geni and uh
3:47
specifically in this cases J geni and uh
3:47
specifically in this cases J geni and uh yeah uh I I have a new book book on that
3:50
yeah uh I I have a new book book on that
3:50
yeah uh I I have a new book book on that topic on securing Azure open AI app
3:52
topic on securing Azure open AI app
3:52
topic on securing Azure open AI app specifically coming up uh early early
3:54
specifically coming up uh early early
3:54
specifically coming up uh early early next year right and and uh we're looking
3:58
next year right and and uh we're looking
3:58
next year right and and uh we're looking forward but I'd like to drill into that
4:00
forward but I'd like to drill into that
4:00
forward but I'd like to drill into that now that we have you here for for a few
4:02
now that we have you here for for a few
4:02
now that we have you here for for a few minutes with us um generative Ai and
4:07
minutes with us um generative Ai and
4:07
minutes with us um generative Ai and security um basically the floor is yours
4:09
security um basically the floor is yours
4:09
security um basically the floor is yours because this is not my field you'll have
4:11
because this is not my field you'll have
4:11
because this is not my field you'll have to you'll have to steer I'll just be
4:14
to you'll have to steer I'll just be
4:14
to you'll have to steer I'll just be yeah for for sure for sure we are We Are
4:17
yeah for for sure for sure we are We Are
4:17
yeah for for sure for sure we are We Are all uh experts until proven otherwise
4:20
all uh experts until proven otherwise
4:20
all uh experts until proven otherwise right correct so yeah basically uh We've
4:23
right correct so yeah basically uh We've
4:23
right correct so yeah basically uh We've now pretty much to the door two years
4:26
now pretty much to the door two years
4:26
now pretty much to the door two years into uh since the LA launch of jat gbt
4:30
into uh since the LA launch of jat gbt
4:30
into uh since the LA launch of jat gbt for general public and we've kind of
4:32
for general public and we've kind of
4:32
for general public and we've kind of gone this kind of there was this huge
4:36
gone this kind of there was this huge
4:36
gone this kind of there was this huge huge HP hype cycle of course in the
4:38
huge HP hype cycle of course in the
4:38
huge HP hype cycle of course in the beginning uh as as we all know the tbit
4:41
beginning uh as as we all know the tbit
4:41
beginning uh as as we all know the tbit this was the fastest adopted consumer
4:43
this was the fastest adopted consumer
4:43
this was the fastest adopted consumer product ever 100 million customers or
4:46
product ever 100 million customers or
4:46
product ever 100 million customers or 100 million users because it's it was a
4:48
100 million users because it's it was a
4:48
100 million users because it's it was a free product for a long time in a record
4:50
free product for a long time in a record
4:50
free product for a long time in a record time um very very much hype both on the
4:54
time um very very much hype both on the
4:54
time um very very much hype both on the consumer space as well as the Enterprise
4:57
consumer space as well as the Enterprise
4:57
consumer space as well as the Enterprise space in there this couple of things
5:00
space in there this couple of things
5:00
space in there this couple of things that really happened uh from that well
5:02
that really happened uh from that well
5:02
that really happened uh from that well first of all uh we are now kind of
5:04
first of all uh we are now kind of
5:04
first of all uh we are now kind of getting hit by not hit by the reality of
5:07
getting hit by not hit by the reality of
5:07
getting hit by not hit by the reality of okay if you actually want to get the
5:09
okay if you actually want to get the
5:09
okay if you actually want to get the benefits of that you do need to have a
5:11
benefits of that you do need to have a
5:11
benefits of that you do need to have a proper Cloud environment in place you
5:13
proper Cloud environment in place you
5:13
proper Cloud environment in place you talk talked previously about Landing
5:14
talk talked previously about Landing
5:14
talk talked previously about Landing zones and these other items in here
5:16
zones and these other items in here
5:16
zones and these other items in here before but at the same time you also
5:18
before but at the same time you also
5:19
before but at the same time you also need to to get the most out of it you
5:21
need to to get the most out of it you
5:21
need to to get the most out of it you need to ground those applications with
5:23
need to ground those applications with
5:23
need to ground those applications with your crown jewels the data that's that's
5:25
your crown jewels the data that's that's
5:25
your crown jewels the data that's that's kind of usually locked uh behind you
5:28
kind of usually locked uh behind you
5:28
kind of usually locked uh behind you know a vault and a key in your
5:30
know a vault and a key in your
5:30
know a vault and a key in your Enterprise kind of on premises
5:32
Enterprise kind of on premises
5:32
Enterprise kind of on premises environments maybe in your Erp systems
5:34
environments maybe in your Erp systems
5:34
environments maybe in your Erp systems etc etc so you basically do all of that
5:38
etc etc so you basically do all of that
5:38
etc etc so you basically do all of that work before you can actually get the
5:40
work before you can actually get the
5:40
work before you can actually get the benefits of of
5:42
benefits of of
5:42
benefits of of ji yeah and at the end of the day the
5:46
ji yeah and at the end of the day the
5:46
ji yeah and at the end of the day the only thing that the bad guys want is
5:49
only thing that the bad guys want is
5:49
only thing that the bad guys want is your data right and and and now you're
5:51
your data right and and and now you're
5:51
your data right and and and now you're basically putting a prompt in front of
5:53
basically putting a prompt in front of
5:53
basically putting a prompt in front of it and just like ask me
5:55
it and just like ask me
5:55
it and just like ask me anything exactly so so we have the the
5:58
anything exactly so so we have the the
5:58
anything exactly so so we have the the old stuff that even even if you even if
6:00
old stuff that even even if you even if
6:00
old stuff that even even if you even if you're are just dealing with a kind of
6:02
you're are just dealing with a kind of
6:02
you're are just dealing with a kind of application that we already know for
6:04
application that we already know for
6:04
application that we already know for kind of general you know uh crude
6:06
kind of general you know uh crude
6:06
kind of general you know uh crude application you know an application that
6:08
application you know an application that
6:08
application you know an application that kind of lets us have have a form against
6:11
kind of lets us have have a form against
6:11
kind of lets us have have a form against a a kind of wellestablished typee of
6:12
a a kind of wellestablished typee of
6:12
a a kind of wellestablished typee of database that we can read and write and
6:15
database that we can read and write and
6:15
database that we can read and write and and and delete some of that data even
6:17
and and delete some of that data even
6:17
and and delete some of that data even that is not kind of a fully solved
6:19
that is not kind of a fully solved
6:19
that is not kind of a fully solved problem you and I been working on on
6:20
problem you and I been working on on
6:21
problem you and I been working on on cloud security for a long time there's
6:22
cloud security for a long time there's
6:22
cloud security for a long time there's still a lot of things that that can go
6:24
still a lot of things that that can go
6:24
still a lot of things that that can go wrong in there and now we are doing that
6:26
wrong in there and now we are doing that
6:26
wrong in there and now we are doing that maybe some of some companies are kind of
6:28
maybe some of some companies are kind of
6:28
maybe some of some companies are kind of going going at with trying to cut some
6:31
going going at with trying to cut some
6:31
going going at with trying to cut some Corners uh they need to do all of those
6:33
Corners uh they need to do all of those
6:33
Corners uh they need to do all of those kind of technical depth Parts at the
6:35
kind of technical depth Parts at the
6:35
kind of technical depth Parts at the same time as doing that uh for Gen which
6:38
same time as doing that uh for Gen which
6:38
same time as doing that uh for Gen which also introduces new risks not just the
6:40
also introduces new risks not just the
6:40
also introduces new risks not just the established risks that we had so we've
6:42
established risks that we had so we've
6:42
established risks that we had so we've even had already some of these cases uh
6:45
even had already some of these cases uh
6:45
even had already some of these cases uh in the public where um for example
6:47
in the public where um for example
6:47
in the public where um for example there's Air Canada had had an had a bot
6:51
there's Air Canada had had an had a bot
6:51
there's Air Canada had had an had a bot that was grounded on their data but that
6:53
that was grounded on their data but that
6:53
that was grounded on their data but that produces produced hallucinations and
6:56
produces produced hallucinations and
6:56
produces produced hallucinations and because it was uh presenting itself as a
6:59
because it was uh presenting itself as a
6:59
because it was uh presenting itself as a member of the of the of the of the
7:03
member of the of the of the of the
7:03
member of the of the of the of the airline they they basically needed to
7:05
airline they they basically needed to
7:05
airline they they basically needed to honor any anything that was made up by
7:08
honor any anything that was made up by
7:08
honor any anything that was made up by by that bot and there was a specific
7:10
by that bot and there was a specific
7:10
by that bot and there was a specific case that got a lot of publicity there
7:12
case that got a lot of publicity there
7:12
case that got a lot of publicity there where they made up uh where the bot made
7:14
where they made up uh where the bot made
7:14
where they made up uh where the bot made up basically a
7:17
up basically a
7:17
up basically a um a a better policy for refundability
7:21
um a a better policy for refundability
7:21
um a a better policy for refundability in case of you know illness in family
7:24
in case of you know illness in family
7:24
in case of you know illness in family than was actually the case and the
7:26
than was actually the case and the
7:26
than was actually the case and the person was never never able to get their
7:28
person was never never able to get their
7:28
person was never never able to get their re refund
7:30
re refund
7:30
re refund until they they sued the company and and
7:33
until they they sued the company and and
7:33
until they they sued the company and and they they were forced the company was
7:34
they they were forced the company was
7:34
they they were forced the company was forced to honor that commitment there so
7:36
forced to honor that commitment there so
7:36
forced to honor that commitment there so this is kind of a new type of kind of
7:38
this is kind of a new type of kind of
7:38
this is kind of a new type of kind of business threat that we are starting to
7:40
business threat that we are starting to
7:40
business threat that we are starting to see and at the same time also as you
7:42
see and at the same time also as you
7:42
see and at the same time also as you mentioned like prompt injections
7:43
mentioned like prompt injections
7:43
mentioned like prompt injections completely new type of uh of attack
7:46
completely new type of uh of attack
7:46
completely new type of uh of attack Vector uh no longer just even
7:48
Vector uh no longer just even
7:48
Vector uh no longer just even application Level but kind of on the
7:49
application Level but kind of on the
7:49
application Level but kind of on the content level what are you talking
7:51
content level what are you talking
7:51
content level what are you talking talking about not on the protocol or
7:53
talking about not on the protocol or
7:53
talking about not on the protocol or level but actually on the content level
7:55
level but actually on the content level
7:55
level but actually on the content level there uh we start we starting to see uh
7:58
there uh we start we starting to see uh
7:58
there uh we start we starting to see uh not just theore Ally but but in practice
8:01
not just theore Ally but but in practice
8:01
not just theore Ally but but in practice already this also this these sort of uh
8:04
already this also this these sort of uh
8:04
already this also this these sort of uh injections into the system breaking
8:06
injections into the system breaking
8:06
injections into the system breaking outside of these C drils that we are
8:08
outside of these C drils that we are
8:08
outside of these C drils that we are supposedly getting from uh from these
8:10
supposedly getting from uh from these
8:10
supposedly getting from uh from these vendors and that of course leads to even
8:14
vendors and that of course leads to even
8:14
vendors and that of course leads to even even more fascinating new types of
8:16
even more fascinating new types of
8:16
even more fascinating new types of threats in gen as well it really does
8:19
threats in gen as well it really does
8:19
threats in gen as well it really does and so given that uh you were you were
8:22
and so given that uh you were you were
8:22
and so given that uh you were you were alluding to the the tools that the the
8:24
alluding to the the tools that the the
8:24
alluding to the the tools that the the guard rails that we're supposed to have
8:25
guard rails that we're supposed to have
8:25
guard rails that we're supposed to have from the vendors um there are tools and
8:29
from the vendors um there are tools and
8:29
from the vendors um there are tools and things that we we should be responsibly
8:32
things that we we should be responsibly
8:32
things that we we should be responsibly putting in place when we uh when we put
8:34
putting in place when we uh when we put
8:34
putting in place when we uh when we put out whatever a bot or or or something
8:37
out whatever a bot or or or something
8:37
out whatever a bot or or or something that that anyone can interact with even
8:39
that that anyone can interact with even
8:39
that that anyone can interact with even if it's internal right it still has
8:41
if it's internal right it still has
8:41
if it's internal right it still has security issues you can have it divulge
8:44
security issues you can have it divulge
8:44
security issues you can have it divulge uh sensitive information and things like
8:46
uh sensitive information and things like
8:46
uh sensitive information and things like that unless you do the right thing so
8:49
that unless you do the right thing so
8:49
that unless you do the right thing so tell us about the tools that we're
8:50
tell us about the tools that we're
8:50
tell us about the tools that we're supposed to be using to to ensure or you
8:53
supposed to be using to to ensure or you
8:53
supposed to be using to to ensure or you know to have a chance to have security
8:55
know to have a chance to have security
8:55
know to have a chance to have security with our generative AI yeah yeah uh so
8:58
with our generative AI yeah yeah uh so
8:58
with our generative AI yeah yeah uh so so very brief there's there's multiple
9:00
so very brief there's there's multiple
9:00
so very brief there's there's multiple different definitions for whatever is a
9:03
different definitions for whatever is a
9:03
different definitions for whatever is a generative AI application but at this
9:06
generative AI application but at this
9:06
generative AI application but at this very core you can think of it from the
9:08
very core you can think of it from the
9:08
very core you can think of it from the perspective of a kind of very familiar
9:09
perspective of a kind of very familiar
9:09
perspective of a kind of very familiar kind of threee application uh the front
9:12
kind of threee application uh the front
9:12
kind of threee application uh the front end part of the application just happens
9:14
end part of the application just happens
9:14
end part of the application just happens to be not just a a UI that that a user
9:17
to be not just a a UI that that a user
9:17
to be not just a a UI that that a user can click through but it is actually
9:20
can click through but it is actually
9:20
can click through but it is actually more um more interactive it will evolve
9:23
more um more interactive it will evolve
9:23
more um more interactive it will evolve over time it's h it both learns from
9:26
over time it's h it both learns from
9:26
over time it's h it both learns from from the context of the user but also
9:28
from the context of the user but also
9:28
from the context of the user but also the the models are behind that natural
9:30
the the models are behind that natural
9:30
the the models are behind that natural language part of the llm model they will
9:33
language part of the llm model they will
9:33
language part of the llm model they will also evolve so uh so the different uh so
9:37
also evolve so uh so the different uh so
9:37
also evolve so uh so the different uh so so even if you don't do anything when
9:38
so even if you don't do anything when
9:38
so even if you don't do anything when you ship something to public uh to uh to
9:41
you ship something to public uh to uh to
9:41
you ship something to public uh to uh to production you you you run all of your
9:43
production you you you run all of your
9:43
production you you you run all of your tests if you just leave it be uh the
9:46
tests if you just leave it be uh the
9:46
tests if you just leave it be uh the content will be different from uh from
9:48
content will be different from uh from
9:48
content will be different from uh from now compared to six months ago and it
9:51
now compared to six months ago and it
9:51
now compared to six months ago and it might behave differently it might even
9:53
might behave differently it might even
9:53
might behave differently it might even just break itself uh if if you don't if
9:55
just break itself uh if if you don't if
9:55
just break itself uh if if you don't if you're not paying enough attention there
9:58
you're not paying enough attention there
9:58
you're not paying enough attention there but in so so it is a very different
10:01
but in so so it is a very different
10:01
but in so so it is a very different different approach uh to to security it
10:03
different approach uh to to security it
10:03
different approach uh to to security it needs to be much more continuous but in
10:05
needs to be much more continuous but in
10:05
needs to be much more continuous but in general we have exactly the same set of
10:07
general we have exactly the same set of
10:07
general we have exactly the same set of tools that we had before uh we need to
10:09
tools that we had before uh we need to
10:10
tools that we had before uh we need to focus on audit logging we need to focus
10:12
focus on audit logging we need to focus
10:12
focus on audit logging we need to focus on controlling access especially if our
10:14
on controlling access especially if our
10:14
on controlling access especially if our llm is grounded with with data and is
10:17
llm is grounded with with data and is
10:17
llm is grounded with with data and is even maybe approaching this in a more
10:19
even maybe approaching this in a more
10:19
even maybe approaching this in a more agend way meaning that the llm talks to
10:22
agend way meaning that the llm talks to
10:22
agend way meaning that the llm talks to another llm which talks to another LM
10:24
another llm which talks to another LM
10:24
another llm which talks to another LM which maybe talks to an API the access
10:26
which maybe talks to an API the access
10:26
which maybe talks to an API the access control is much more complex and it can
10:29
control is much more complex and it can
10:29
control is much more complex and it can result in much uh kind of broader
10:32
result in much uh kind of broader
10:32
result in much uh kind of broader implication than just accidentally
10:35
implication than just accidentally
10:35
implication than just accidentally publishing something that of data that
10:37
publishing something that of data that
10:37
publishing something that of data that you want but maybe it's also a
10:38
you want but maybe it's also a
10:38
you want but maybe it's also a combination of your yourself and some
10:40
combination of your yourself and some
10:40
combination of your yourself and some suppliers there and then kind of the
10:42
suppliers there and then kind of the
10:42
suppliers there and then kind of the regular stuff of adding network security
10:44
regular stuff of adding network security
10:44
regular stuff of adding network security controls and control plane security
10:46
controls and control plane security
10:46
controls and control plane security these are all of the things that we know
10:48
these are all of the things that we know
10:48
these are all of the things that we know and we should we we can uh put in place
10:52
and we should we we can uh put in place
10:52
and we should we we can uh put in place not all of those tools are kind of in
10:54
not all of those tools are kind of in
10:54
not all of those tools are kind of in general availability level of maturity
10:56
general availability level of maturity
10:56
general availability level of maturity there's some hits and misses in there
10:58
there's some hits and misses in there
10:58
there's some hits and misses in there when when it comes to those products on
11:00
when when it comes to those products on
11:00
when when it comes to those products on the Microsoft side on the Azure open AI
11:02
the Microsoft side on the Azure open AI
11:02
the Microsoft side on the Azure open AI side we are getting into quite a good
11:04
side we are getting into quite a good
11:04
side we are getting into quite a good level but for example there's still
11:06
level but for example there's still
11:06
level but for example there's still still some policies that are missing you
11:07
still some policies that are missing you
11:07
still some policies that are missing you know there's still some catching up to
11:09
know there's still some catching up to
11:09
know there's still some catching up to do over there but then we also have
11:11
do over there but then we also have
11:11
do over there but then we also have these new types of security control if I
11:13
these new types of security control if I
11:13
these new types of security control if I mentioned the term guard rail in there
11:16
mentioned the term guard rail in there
11:16
mentioned the term guard rail in there that's that's basically when we used to
11:17
that's that's basically when we used to
11:17
that's that's basically when we used to have a web application firewall uh which
11:20
have a web application firewall uh which
11:20
have a web application firewall uh which was kind of a de facto uh way of
11:23
was kind of a de facto uh way of
11:23
was kind of a de facto uh way of protecting your front front end now in
11:25
protecting your front front end now in
11:25
protecting your front front end now in addition to that web application
11:27
addition to that web application
11:27
addition to that web application Fireball between your mod and your
11:30
Fireball between your mod and your
11:30
Fireball between your mod and your client from the from the interaction
11:32
client from the from the interaction
11:32
client from the from the interaction perspective you can put this crail or
11:34
perspective you can put this crail or
11:34
perspective you can put this crail or Fireball to protect both the input from
11:36
Fireball to protect both the input from
11:37
Fireball to protect both the input from the user to the model as well as the
11:39
the user to the model as well as the
11:39
the user to the model as well as the output from the uh from the model to the
11:41
output from the uh from the model to the
11:41
output from the uh from the model to the user so prompt injections from will be
11:44
user so prompt injections from will be
11:44
user so prompt injections from will be that input production and uh kind of
11:46
that input production and uh kind of
11:46
that input production and uh kind of malicious behavior uh you know swearing
11:50
malicious behavior uh you know swearing
11:50
malicious behavior uh you know swearing uh abusive messaging that will be uh
11:52
uh abusive messaging that will be uh
11:52
uh abusive messaging that will be uh output protection uh from that fireball
11:55
output protection uh from that fireball
11:55
output protection uh from that fireball or from that guardrail perspective there
11:58
or from that guardrail perspective there
11:58
or from that guardrail perspective there yeah yeah and and what are the I mean
12:01
yeah yeah and and what are the I mean
12:01
yeah yeah and and what are the I mean it's maybe a trivial question but what
12:04
it's maybe a trivial question but what
12:04
it's maybe a trivial question but what are the stakes here in terms of you know
12:08
are the stakes here in terms of you know
12:08
are the stakes here in terms of you know getting this wrong it's it's I guess
12:09
getting this wrong it's it's I guess
12:09
getting this wrong it's it's I guess maybe the same as always it's it's a lot
12:12
maybe the same as always it's it's a lot
12:12
maybe the same as always it's it's a lot of business value and other things at
12:14
of business value and other things at
12:14
of business value and other things at stake yeah AB absolutely uh I think the
12:17
stake yeah AB absolutely uh I think the
12:17
stake yeah AB absolutely uh I think the the big part is that it's no longer just
12:19
the big part is that it's no longer just
12:19
the big part is that it's no longer just whatever you happen to have on the cloud
12:22
whatever you happen to have on the cloud
12:22
whatever you happen to have on the cloud but most of these Enterprise grade llm
12:25
but most of these Enterprise grade llm
12:25
but most of these Enterprise grade llm uh deployments or llm Transformations
12:28
uh deployments or llm Transformations
12:28
uh deployments or llm Transformations are actually building up up on top of
12:33
are actually building up up on top of
12:33
are actually building up up on top of basically all of all of the crown juels
12:35
basically all of all of the crown juels
12:35
basically all of all of the crown juels of your data that I men there before so
12:38
of your data that I men there before so
12:38
of your data that I men there before so it's not not just that you have a
12:39
it's not not just that you have a
12:39
it's not not just that you have a storage account somewhere that gets
12:41
storage account somewhere that gets
12:41
storage account somewhere that gets accidentally exposed uh you have that
12:44
accidentally exposed uh you have that
12:44
accidentally exposed uh you have that and kind of unlimited uh way of of
12:47
and kind of unlimited uh way of of
12:47
and kind of unlimited uh way of of talking to that storage account and that
12:49
talking to that storage account and that
12:49
talking to that storage account and that storage account happens to have been
12:51
storage account happens to have been
12:51
storage account happens to have been cross correlated with uh with everything
12:54
cross correlated with uh with everything
12:54
cross correlated with uh with everything that you have on premises as as well so
12:56
that you have on premises as as well so
12:56
that you have on premises as as well so it's kind of adding that adding to that
12:58
it's kind of adding that adding to that
12:58
it's kind of adding that adding to that impact of that particular loss of
13:01
impact of that particular loss of
13:01
impact of that particular loss of private confidential data for example so
13:03
private confidential data for example so
13:03
private confidential data for example so the impact can be
13:05
the impact can be
13:05
the impact can be total it can be all all right well so um
13:10
total it can be all all right well so um
13:10
total it can be all all right well so um looking at the market I I know that you
13:12
looking at the market I I know that you
13:12
looking at the market I I know that you and and with within your company and and
13:14
and and with within your company and and
13:14
and and with within your company and and the work that you do you see a lot of
13:17
the work that you do you see a lot of
13:17
the work that you do you see a lot of cases um so uh what is like there there
13:22
cases um so uh what is like there there
13:22
cases um so uh what is like there there ought to be a problem security wise and
13:25
ought to be a problem security wise and
13:25
ought to be a problem security wise and and generative a uh context when when
13:29
and generative a uh context when when
13:29
and generative a uh context when when we're talking about um your um uh the
13:35
we're talking about um your um uh the
13:35
we're talking about um your um uh the maturity right there is a huge hype and
13:37
maturity right there is a huge hype and
13:37
maturity right there is a huge hype and everybody should have product our we
13:40
everybody should have product our we
13:40
everybody should have product our we before we had product P now we we need
13:42
before we had product P now we we need
13:42
before we had product P now we we need to have product P plus AI right just
13:44
to have product P plus AI right just
13:44
to have product P plus AI right just because right because the hype says so
13:47
because right because the hype says so
13:47
because right because the hype says so but I I'm assuming a lot of companies
13:49
but I I'm assuming a lot of companies
13:49
but I I'm assuming a lot of companies are not ready for this and and the
13:51
are not ready for this and and the
13:51
are not ready for this and and the security implications of this have to be
13:54
security implications of this have to be
13:54
security implications of this have to be grave Absol absolutely I I think we
13:57
grave Absol absolutely I I think we
13:57
grave Absol absolutely I I think we we've already passed kind of the biggest
14:00
we've already passed kind of the biggest
14:00
we've already passed kind of the biggest uh kind of mistakes on on that area
14:02
uh kind of mistakes on on that area
14:02
uh kind of mistakes on on that area certainly we haven't passed the the the
14:04
certainly we haven't passed the the the
14:04
certainly we haven't passed the the the biggest volume on that we are if if
14:07
biggest volume on that we are if if
14:07
biggest volume on that we are if if cloud is maybe let's say that half of
14:10
cloud is maybe let's say that half of
14:10
cloud is maybe let's say that half of the companies are still yet to do their
14:12
the companies are still yet to do their
14:12
the companies are still yet to do their big migration to the cloud just you know
14:14
big migration to the cloud just you know
14:14
big migration to the cloud just you know coming up probably true or more right
14:17
coming up probably true or more right
14:17
coming up probably true or more right it's yeah so we are not even halfway
14:19
it's yeah so we are not even halfway
14:19
it's yeah so we are not even halfway true on on J I'm I'm sure we are maybe
14:22
true on on J I'm I'm sure we are maybe
14:22
true on on J I'm I'm sure we are maybe like 10% uh true so to say but the the
14:27
like 10% uh true so to say but the the
14:27
like 10% uh true so to say but the the kind of R&D or this kind of uh playing
14:31
kind of R&D or this kind of uh playing
14:31
kind of R&D or this kind of uh playing out with things not just going full in
14:33
out with things not just going full in
14:33
out with things not just going full in is uh is is much broader than with the
14:36
is uh is is much broader than with the
14:36
is uh is is much broader than with the cloud because there is inherent business
14:38
cloud because there is inherent business
14:38
cloud because there is inherent business value uh in in there as well so it's
14:41
value uh in in there as well so it's
14:41
value uh in in there as well so it's it's not just not just the kind of it
14:44
it's not just not just the kind of it
14:44
it's not just not just the kind of it infrastructure move that we will be
14:45
infrastructure move that we will be
14:45
infrastructure move that we will be talking about but it's also uh not even
14:48
talking about but it's also uh not even
14:48
talking about but it's also uh not even your even your data team that that will
14:50
your even your data team that that will
14:50
your even your data team that that will be focused on that but you might have um
14:53
be focused on that but you might have um
14:53
be focused on that but you might have um your end users just starting to consume
14:57
your end users just starting to consume
14:57
your end users just starting to consume generative AI on top of your data even
15:00
generative AI on top of your data even
15:00
generative AI on top of your data even using consumer services let let alone
15:03
using consumer services let let alone
15:03
using consumer services let let alone your internal products so this kind of
15:05
your internal products so this kind of
15:05
your internal products so this kind of same thing that we had for software
15:06
same thing that we had for software
15:07
same thing that we had for software service overall we had shadow shadow it
15:10
service overall we had shadow shadow it
15:10
service overall we had shadow shadow it coming in uh for for like you will
15:12
coming in uh for for like you will
15:12
coming in uh for for like you will instead of using your inter internal
15:15
instead of using your inter internal
15:15
instead of using your inter internal file sharing tool that that may be
15:16
file sharing tool that that may be
15:16
file sharing tool that that may be sanctioned from the company like you
15:18
sanctioned from the company like you
15:18
sanctioned from the company like you would use like Dropbox or past pin or
15:20
would use like Dropbox or past pin or
15:20
would use like Dropbox or past pin or whatever and sometimes sometimes data
15:23
whatever and sometimes sometimes data
15:23
whatever and sometimes sometimes data will leak through that the same is true
15:25
will leak through that the same is true
15:25
will leak through that the same is true with Gen and most of the cases that that
15:28
with Gen and most of the cases that that
15:28
with Gen and most of the cases that that we can talk publicly about and we we
15:30
we can talk publicly about and we we
15:30
we can talk publicly about and we we have examples of are about these cases
15:32
have examples of are about these cases
15:32
have examples of are about these cases when someone's using um kind of consumer
15:35
when someone's using um kind of consumer
15:35
when someone's using um kind of consumer grade gen tools uh with Enterprise data
15:39
grade gen tools uh with Enterprise data
15:39
grade gen tools uh with Enterprise data and that's kind of the big behavioral
15:41
and that's kind of the big behavioral
15:41
and that's kind of the big behavioral change that we certainly need to need to
15:43
change that we certainly need to need to
15:43
change that we certainly need to need to make first but this kind of Customs
15:46
make first but this kind of Customs
15:46
make first but this kind of Customs custom application so the real
15:48
custom application so the real
15:48
custom application so the real generative AI part that that we still
15:50
generative AI part that that we still
15:50
generative AI part that that we still need to move towards that requires
15:52
need to move towards that requires
15:52
need to move towards that requires building on top of an existing uh
15:55
building on top of an existing uh
15:55
building on top of an existing uh existing uh gen gen platform uh most of
15:59
existing uh gen gen platform uh most of
15:59
existing uh gen gen platform uh most of the cases then there are of course some
16:01
the cases then there are of course some
16:01
the cases then there are of course some product companies that would build their
16:03
product companies that would build their
16:03
product companies that would build their own models and kind of even even go far
16:06
own models and kind of even even go far
16:06
own models and kind of even even go far go as far as you know not not using EST
16:10
go as far as you know not not using EST
16:10
go as far as you know not not using EST established uh model models that are
16:12
established uh model models that are
16:12
established uh model models that are hosted in the cloud but would host their
16:14
hosted in the cloud but would host their
16:14
hosted in the cloud but would host their own models will tune tune them uh maybe
16:17
own models will tune tune them uh maybe
16:17
own models will tune tune them uh maybe even uh contribute to open source as
16:20
even uh contribute to open source as
16:20
even uh contribute to open source as some of some of them do that's kind of a
16:22
some of some of them do that's kind of a
16:22
some of some of them do that's kind of a whole other
16:24
whole other
16:24
whole other space yeah
16:26
space yeah
16:26
space yeah definitely so so as a as a fin fin thing
16:29
definitely so so as a as a fin fin thing
16:29
definitely so so as a as a fin fin thing maybe for this episode it's it's been
16:31
maybe for this episode it's it's been
16:31
maybe for this episode it's it's been really fascinating so far but do you
16:33
really fascinating so far but do you
16:33
really fascinating so far but do you have any kind
16:35
have any kind
16:35
have any kind of like what's your best type of like if
16:38
of like what's your best type of like if
16:38
of like what's your best type of like if you have a couple of quick tips advice
16:41
you have a couple of quick tips advice
16:41
you have a couple of quick tips advice like what what should should or should a
16:43
like what what should should or should a
16:43
like what what should should or should a company not do when embarking on this AI
16:46
company not do when embarking on this AI
16:46
company not do when embarking on this AI Adventure uh generative AI we need it in
16:48
Adventure uh generative AI we need it in
16:49
Adventure uh generative AI we need it in our product and and security is is is
16:51
our product and and security is is is
16:51
our product and and security is is is important what what should you tell them
16:54
important what what should you tell them
16:54
important what what should you tell them to absolutely do or do not absolutely I
16:57
to absolutely do or do not absolutely I
16:57
to absolutely do or do not absolutely I I think Tre three things that are really
17:00
I think Tre three things that are really
17:00
I think Tre three things that are really applicable for everyone is first of all
17:03
applicable for everyone is first of all
17:03
applicable for everyone is first of all really you know limit access to that
17:05
really you know limit access to that
17:05
really you know limit access to that consumer consumer grade there's many
17:08
consumer consumer grade there's many
17:08
consumer consumer grade there's many different versions of doing that it may
17:10
different versions of doing that it may
17:10
different versions of doing that it may be kind of adding you know something to
17:12
be kind of adding you know something to
17:12
be kind of adding you know something to your firewalls or something something to
17:14
your firewalls or something something to
17:14
your firewalls or something something to your company devices that kind of
17:16
your company devices that kind of
17:16
your company devices that kind of detects and prevents access to those
17:18
detects and prevents access to those
17:18
detects and prevents access to those sites there's kind of that can be kind
17:19
sites there's kind of that can be kind
17:19
sites there's kind of that can be kind of Never Ending game it can be you need
17:21
of Never Ending game it can be you need
17:21
of Never Ending game it can be you need a policy in place uh or you need to just
17:23
a policy in place uh or you need to just
17:24
a policy in place uh or you need to just educate people that's that's number one
17:26
educate people that's that's number one
17:26
educate people that's that's number one in there uh if you want to want to kind
17:29
in there uh if you want to want to kind
17:29
in there uh if you want to want to kind of have this kind of Enterprise gate
17:30
of have this kind of Enterprise gate
17:30
of have this kind of Enterprise gate kind of U Enterprise grade consumer type
17:33
kind of U Enterprise grade consumer type
17:33
kind of U Enterprise grade consumer type of uh or personal productivity use cases
17:36
of uh or personal productivity use cases
17:36
of uh or personal productivity use cases look at look at some consumer uh kind of
17:39
look at look at some consumer uh kind of
17:39
look at look at some consumer uh kind of you look at some some some tools like
17:41
you look at some some some tools like
17:41
you look at some some some tools like co-pilots that kind of provide similar
17:43
co-pilots that kind of provide similar
17:43
co-pilots that kind of provide similar tooling or if you really need to have
17:46
tooling or if you really need to have
17:46
tooling or if you really need to have you know open AI type of thing you can
17:48
you know open AI type of thing you can
17:48
you know open AI type of thing you can host similar things on your yourself
17:51
host similar things on your yourself
17:51
host similar things on your yourself expose that internally available U
17:54
expose that internally available U
17:54
expose that internally available U internally as well at the very least you
17:56
internally as well at the very least you
17:56
internally as well at the very least you should be using your existing kind of
17:58
should be using your existing kind of
17:58
should be using your existing kind of enterprise users in there so limiting
18:00
enterprise users in there so limiting
18:00
enterprise users in there so limiting that access to the consumer grade in
18:02
that access to the consumer grade in
18:02
that access to the consumer grade in there second second of all if you are
18:05
there second second of all if you are
18:05
there second second of all if you are going into building your own then I
18:07
going into building your own then I
18:07
going into building your own then I really recommend uh that making that
18:10
really recommend uh that making that
18:10
really recommend uh that making that into a continuous cycle uh most of the
18:13
into a continuous cycle uh most of the
18:14
into a continuous cycle uh most of the AI World a kind of mlops world is
18:16
AI World a kind of mlops world is
18:16
AI World a kind of mlops world is focused on data data Ops and you're B
18:20
focused on data data Ops and you're B
18:20
focused on data data Ops and you're B barely kind of there when it comes to
18:22
barely kind of there when it comes to
18:22
barely kind of there when it comes to introducing devops capabilities into
18:24
introducing devops capabilities into
18:24
introducing devops capabilities into that world we also need to introduce the
18:27
that world we also need to introduce the
18:27
that world we also need to introduce the silent SEC syllable part of that devc
18:29
silent SEC syllable part of that devc
18:29
silent SEC syllable part of that devc Ops part there into that world as well
18:32
Ops part there into that world as well
18:32
Ops part there into that world as well it needs to be continuous both for
18:34
it needs to be continuous both for
18:34
it needs to be continuous both for testing the kind of uh engineering
18:36
testing the kind of uh engineering
18:36
testing the kind of uh engineering capabilities is the firewall still on
18:39
capabilities is the firewall still on
18:39
capabilities is the firewall still on but also to test uh behavioral changes
18:42
but also to test uh behavioral changes
18:42
but also to test uh behavioral changes um on both the crails these firewalls
18:45
um on both the crails these firewalls
18:45
um on both the crails these firewalls that we we will be using but also on the
18:48
that we we will be using but also on the
18:48
that we we will be using but also on the behavior of our of our model in there
18:50
behavior of our of our model in there
18:50
behavior of our of our model in there right and then the third one was uh
18:53
right and then the third one was uh
18:53
right and then the third one was uh really if you are doing that leverage
18:55
really if you are doing that leverage
18:55
really if you are doing that leverage you know leverage the existing vendors
18:57
you know leverage the existing vendors
18:57
you know leverage the existing vendors and tools that are out there don't try
18:59
and tools that are out there don't try
18:59
and tools that are out there don't try to reinvent the wheel both the models
19:01
to reinvent the wheel both the models
19:01
to reinvent the wheel both the models that are out there are great most of
19:03
that are out there are great most of
19:03
that are out there are great most of your problems business problems can be
19:05
your problems business problems can be
19:05
your problems business problems can be solved with you know rag instead of
19:06
solved with you know rag instead of
19:07
solved with you know rag instead of fine-tuning and most of the time you can
19:09
fine-tuning and most of the time you can
19:09
fine-tuning and most of the time you can uh can and should leverage both the
19:11
uh can and should leverage both the
19:11
uh can and should leverage both the models that are out there available
19:13
models that are out there available
19:13
models that are out there available hosted in the cloud as well as the
19:15
hosted in the cloud as well as the
19:15
hosted in the cloud as well as the guardrails there are open- Source
19:17
guardrails there are open- Source
19:17
guardrails there are open- Source Alternatives and very good ones as well
19:19
Alternatives and very good ones as well
19:20
Alternatives and very good ones as well uh but just putting this together in a
19:22
uh but just putting this together in a
19:22
uh but just putting this together in a hosted fashion is is hard enough of a
19:24
hosted fashion is is hard enough of a
19:24
hosted fashion is is hard enough of a problem to solve so I really recommend
19:26
problem to solve so I really recommend
19:26
problem to solve so I really recommend starting uh starting from there
19:29
starting uh starting from there
19:29
starting uh starting from there fantastic wow what a summary that's
19:31
fantastic wow what a summary that's
19:31
fantastic wow what a summary that's Perfection I would call that a wrap
19:34
Perfection I would call that a wrap
19:34
Perfection I would call that a wrap thank you so much for coming on the
19:35
thank you so much for coming on the
19:35
thank you so much for coming on the cloud Show Carl it's been really really
19:37
cloud Show Carl it's been really really
19:37
cloud Show Carl it's been really really good talking to you thank you so much
19:39
good talking to you thank you so much
19:39
good talking to you thank you so much and audience please join us again next
19:42
and audience please join us again next
19:42
and audience please join us again next week for another episode of the cloud
19:43
week for another episode of the cloud
19:43
week for another episode of the cloud show
19:45
show
19:45
show [Music]
#Business & Industrial
#Computers & Electronics