0:02
hello everyone and welcome to another
0:04
hello everyone and welcome to another
0:04
hello everyone and welcome to another episode of the cloud show I am here at
0:07
episode of the cloud show I am here at
0:07
episode of the cloud show I am here at espc in Stockholm it's a huge conference
0:10
espc in Stockholm it's a huge conference
0:10
espc in Stockholm it's a huge conference and we're having a lot of fun here and I
0:12
and we're having a lot of fun here and I
0:12
and we're having a lot of fun here and I get to meet up with some of my really
0:13
get to meet up with some of my really
0:13
get to meet up with some of my really good friends around the world and talk
0:15
good friends around the world and talk
0:15
good friends around the world and talk about all kinds of interesting topics so
0:18
about all kinds of interesting topics so
0:18
about all kinds of interesting topics so now we're going to delve into talk about
0:21
now we're going to delve into talk about
0:21
now we're going to delve into talk about and try to figure out the return on
0:23
and try to figure out the return on
0:23
and try to figure out the return on investment for the quite significant
0:25
investment for the quite significant
0:25
investment for the quite significant Capital expend expenditure that a uh
0:29
Capital expend expenditure that a uh
0:29
Capital expend expenditure that a uh manager could make
0:30
manager could make while getting co-pilot licenses and to
0:33
while getting co-pilot licenses and to
0:33
while getting co-pilot licenses and to talk about this interesting topic I am
0:35
talk about this interesting topic I am
0:35
talk about this interesting topic I am going today today have the star of Tony
0:39
going today today have the star of Tony
0:39
going today today have the star of Tony [Music]
0:47
Redmond so hello Tony how you doing I'm
0:50
Redmond so hello Tony how you doing I'm
0:51
Redmond so hello Tony how you doing I'm doing fine thank you welcome to the
0:52
doing fine thank you welcome to the
0:52
doing fine thank you welcome to the cloud show thank you yes the espc
0:55
cloud show thank you yes the espc
0:55
cloud show thank you yes the espc conference pretty cool huh yeah I've
0:57
conference pretty cool huh yeah I've
0:57
conference pretty cool huh yeah I've been to a few of them now and uh this is
0:59
been to a few of them now and uh this is
0:59
been to a few of them now and uh this is a very good community conference this is
1:01
a very good community conference this is
1:01
a very good community conference this is a very good Community Conference I agree
1:02
a very good Community Conference I agree
1:02
a very good Community Conference I agree I like the
1:04
I like the um I like the interaction here a lot I
1:08
um I like the interaction here a lot I
1:08
um I like the interaction here a lot I like the level of technical information
1:11
like the level of technical information
1:11
like the level of technical information that's provided by people oh yeah and I
1:13
that's provided by people oh yeah and I
1:13
that's provided by people oh yeah and I think this is a conference that is not
1:14
think this is a conference that is not
1:15
think this is a conference that is not too heavily dominated by Microsoft I
1:17
too heavily dominated by Microsoft I
1:17
too heavily dominated by Microsoft I mean there's clearly a Microsoft
1:19
mean there's clearly a Microsoft
1:19
mean there's clearly a Microsoft presence here but it's not as dominanted
1:21
presence here but it's not as dominanted
1:21
presence here but it's not as dominanted you get as you get some conferen I think
1:23
you get as you get some conferen I think
1:23
you get as you get some conferen I think that there is an open and honest
1:25
that there is an open and honest
1:25
that there is an open and honest conversation about some things that you
1:27
conversation about some things that you
1:27
conversation about some things that you know might be possible to improve in the
1:29
know might be possible to improve in the
1:29
know might be possible to improve in the Microsoft ecosystem as
1:31
Microsoft ecosystem as
1:31
Microsoft ecosystem as well well you got to hope you got to
1:33
well well you got to hope you got to
1:33
well well you got to hope you got to hope I'm not sure that the message gets
1:35
hope I'm not sure that the message gets
1:35
hope I'm not sure that the message gets through all the time but there you have
1:36
through all the time but there you have
1:37
through all the time but there you have that's probably true ongoing challenge
1:39
that's probably true ongoing challenge
1:39
that's probably true ongoing challenge yeah it is it is and and it's a good one
1:41
yeah it is it is and and it's a good one
1:41
yeah it is it is and and it's a good one it's a big Challenge and let's first for
1:43
it's a big Challenge and let's first for
1:43
it's a big Challenge and let's first for the audience uh who are you Tony and
1:45
the audience uh who are you Tony and
1:45
the audience uh who are you Tony and like a little bit what do you
1:47
like a little bit what do you
1:47
like a little bit what do you do well I was a corporate executive for
1:50
do well I was a corporate executive for
1:50
do well I was a corporate executive for years in um digital equipment Compact
1:53
years in um digital equipment Compact
1:54
years in um digital equipment Compact and HP y um I ended up as a vice
1:57
and HP y um I ended up as a vice
1:57
and HP y um I ended up as a vice president in HP a chief technology
1:59
president in HP a chief technology
1:59
president in HP a chief technology officer and uh got to the age of 50
2:02
officer and uh got to the age of 50
2:02
officer and uh got to the age of 50 decided that um maybe there was a
2:05
decided that um maybe there was a
2:05
decided that um maybe there was a different life so I retired and became
2:08
different life so I retired and became
2:08
different life so I retired and became an independent consultant now that's
2:10
an independent consultant now that's
2:10
an independent consultant now that's what I've been doing I write a lot for
2:13
what I've been doing I write a lot for
2:13
what I've been doing I write a lot for about Microsoft 365 mhm uh on practical
2:17
about Microsoft 365 mhm uh on practical
2:17
about Microsoft 365 mhm uh on practical 365.com Office 365 it pros.com and of
2:20
365.com Office 365 it pros.com and of
2:20
365.com Office 365 it pros.com and of course the Office 365 for it Pros book
2:24
course the Office 365 for it Pros book
2:24
course the Office 365 for it Pros book okay and so um you're here at the
2:27
okay and so um you're here at the
2:27
okay and so um you're here at the conference as a speaker yeah and uh you
2:29
conference as a speaker yeah and uh you
2:29
conference as a speaker yeah and uh you you are talking about M365 and so forth
2:32
you are talking about M365 and so forth
2:32
you are talking about M365 and so forth in your talks I guess yep yep very good
2:34
in your talks I guess yep yep very good
2:34
in your talks I guess yep yep very good well I talked about uh the graph po
2:36
well I talked about uh the graph po
2:36
well I talked about uh the graph po shell oh nice SDK uh because we're in a
2:40
shell oh nice SDK uh because we're in a
2:40
shell oh nice SDK uh because we're in a bit of a transition there there's an
2:42
bit of a transition there there's an
2:42
bit of a transition there there's an awful lot of people who came from the on
2:43
awful lot of people who came from the on
2:44
awful lot of people who came from the on premises world who know Powershell I
2:46
premises world who know Powershell I
2:46
premises world who know Powershell I know how to deal with things like
2:47
know how to deal with things like
2:47
know how to deal with things like exchange and SharePoint and team but not
2:51
exchange and SharePoint and team but not
2:51
exchange and SharePoint and team but not necessarily know how to do the graph
2:53
necessarily know how to do the graph
2:53
necessarily know how to do the graph graph is a that was a fun one I did
2:56
graph is a that was a fun one I did
2:56
graph is a that was a fun one I did another session uh which is how to get
2:59
another session uh which is how to get
2:59
another session uh which is how to get information out of the Microsoft 365
3:01
information out of the Microsoft 365
3:01
information out of the Microsoft 365 audit log that's uh that's kind of
3:04
audit log that's uh that's kind of
3:04
audit log that's uh that's kind of interesting because the audit log is the
3:07
interesting because the audit log is the
3:07
interesting because the audit log is the truth of what people actually do inside
3:09
truth of what people actually do inside
3:09
truth of what people actually do inside what they do specifically yes yeah so
3:11
what they do specifically yes yeah so
3:11
what they do specifically yes yeah so that if you understand how to use the
3:13
that if you understand how to use the
3:13
that if you understand how to use the audit log you can extract a lot of very
3:16
audit log you can extract a lot of very
3:16
audit log you can extract a lot of very interesting information from it I I I
3:19
interesting information from it I I I
3:19
interesting information from it I I I believe so and and uh with compliancy
3:21
believe so and and uh with compliancy
3:21
believe so and and uh with compliancy and things like that sometimes companies
3:23
and things like that sometimes companies
3:23
and things like that sometimes companies are they need to they need to have uh
3:26
are they need to they need to have uh
3:26
are they need to they need to have uh proof of proof of audit trails and
3:28
proof of proof of audit trails and
3:28
proof of proof of audit trails and things like that well I mean there's
3:29
things like that well I mean there's
3:29
things like that well I mean there's there other compliance features that are
3:31
there other compliance features that are
3:31
there other compliance features that are available to do that kind of thing uh
3:33
available to do that kind of thing uh
3:34
available to do that kind of thing uh certainly the audit log is of interest
3:35
certainly the audit log is of interest
3:35
certainly the audit log is of interest to forensic in investigators right who
3:38
to forensic in investigators right who
3:38
to forensic in investigators right who need to understand what exactly happened
3:40
need to understand what exactly happened
3:40
need to understand what exactly happened and who did what when and where but uh
3:43
and who did what when and where but uh
3:43
and who did what when and where but uh my perspective on it is that any tenant
3:47
my perspective on it is that any tenant
3:47
my perspective on it is that any tenant administrator needs to know how to get
3:48
administrator needs to know how to get
3:48
administrator needs to know how to get data out of the audit log right just to
3:51
data out of the audit log right just to
3:51
data out of the audit log right just to understand stuff okay about their tenant
3:53
understand stuff okay about their tenant
3:53
understand stuff okay about their tenant very cool so we're almost talking about
3:55
very cool so we're almost talking about
3:55
very cool so we're almost talking about a different thing right now than what we
3:57
a different thing right now than what we
3:57
a different thing right now than what we intended to talk about with this man
4:00
intended to talk about with this man
4:00
intended to talk about with this man we're not we're not because because uh
4:03
we're not we're not because because uh
4:03
we're not we're not because because uh you started off by talking about the
4:06
you started off by talking about the
4:06
you started off by talking about the massive investment that people are
4:07
massive investment that people are
4:07
massive investment that people are making in AI at the moment they are
4:09
making in AI at the moment they are
4:09
making in AI at the moment they are clearly Microsoft has a huge play here
4:11
clearly Microsoft has a huge play here
4:11
clearly Microsoft has a huge play here they're investing up to 20 billion
4:14
they're investing up to 20 billion
4:14
they're investing up to 20 billion massive amounts in that yes in building
4:17
massive amounts in that yes in building
4:17
massive amounts in that yes in building out their data centers and and their
4:19
out their data centers and and their
4:19
out their data centers and and their software yes that cost has been passed
4:22
software yes that cost has been passed
4:22
software yes that cost has been passed on to customers 360 bucks a year for
4:25
on to customers 360 bucks a year for
4:25
on to customers 360 bucks a year for co-pilot mof 365 code that's right so
4:28
co-pilot mof 365 code that's right so
4:28
co-pilot mof 365 code that's right so the question is how do you what data
4:30
the question is how do you what data
4:30
the question is how do you what data exist to
4:32
exist to prove that co-pilot is being used that's
4:35
prove that co-pilot is being used that's
4:35
prove that co-pilot is being used that's right so uh a manager is um you know
4:38
right so uh a manager is um you know
4:38
right so uh a manager is um you know convinced somehow by a salesperson right
4:40
convinced somehow by a salesperson right
4:40
convinced somehow by a salesperson right to to to acquire licenses right for for
4:43
to to to acquire licenses right for for
4:43
to to to acquire licenses right for for co-pilot and it's an individual like per
4:46
co-pilot and it's an individual like per
4:46
co-pilot and it's an individual like per person license per month racks up to
4:49
person license per month racks up to
4:49
person license per month racks up to $360 $60 a year US list per person yeah
4:54
$360 $60 a year US list per person yeah
4:54
$360 $60 a year US list per person yeah so if you have many people then you got
4:56
so if you have many people then you got
4:56
so if you have many people then you got a half the amount um yeah well I mean
4:59
a half the amount um yeah well I mean
4:59
a half the amount um yeah well I mean know I assume that you've got some
5:01
know I assume that you've got some
5:01
know I assume that you've got some negotiation skills and you can get the
5:03
negotiation skills and you can get the
5:03
negotiation skills and you can get the price out but the point is but anyway
5:05
price out but the point is but anyway
5:05
price out but the point is but anyway it's a lot of money it's a big
5:06
it's a lot of money it's a big
5:06
it's a lot of money it's a big investment and the question is so the
5:08
investment and the question is so the
5:08
investment and the question is so the question I get asked a lot by people
5:11
question I get asked a lot by people
5:11
question I get asked a lot by people is okay I've made the investment it's
5:14
is okay I've made the investment it's
5:14
is okay I've made the investment it's about a year into the co-pilot Journey
5:16
about a year into the co-pilot Journey
5:17
about a year into the co-pilot Journey mhm y it's a good time after a year to
5:21
mhm y it's a good time after a year to
5:21
mhm y it's a good time after a year to look back and say to take who's using it
5:24
look back and say to take who's using it
5:24
look back and say to take who's using it yeah what are they using it for that's
5:26
yeah what are they using it for that's
5:26
yeah what are they using it for that's right and am I getting value about it
5:27
right and am I getting value about it
5:28
right and am I getting value about it because you the great promise of AI is
5:30
because you the great promise of AI is
5:30
because you the great promise of AI is that you get uh information faster
5:33
that you get uh information faster
5:33
that you get uh information faster productivity game I'm not absolutely
5:36
productivity game I'm not absolutely
5:36
productivity game I'm not absolutely sure that you get it okay okay I really
5:39
sure that you get it okay okay I really
5:39
sure that you get it okay okay I really haven't I mean I use co-pilot myself but
5:42
haven't I mean I use co-pilot myself but
5:42
haven't I mean I use co-pilot myself but I'm not sure I get that I I get huge
5:44
I'm not sure I get that I I get huge
5:45
I'm not sure I get that I I get huge value out of get of cop I I'm pretty
5:47
value out of get of cop I I'm pretty
5:47
value out of get of cop I I'm pretty sure that some persons in a company can
5:51
sure that some persons in a company can
5:51
sure that some persons in a company can uh return that investment uh with with
5:55
uh return that investment uh with with
5:55
uh return that investment uh with with with what they do right but I'm pretty
5:57
with what they do right but I'm pretty
5:57
with what they do right but I'm pretty convinced that not everyone in the
5:59
convinced that not everyone in the
5:59
convinced that not everyone in the company will be able to do it so the
6:01
company will be able to do it so the
6:01
company will be able to do it so the question is where is the break even if
6:03
question is where is the break even if
6:03
question is where is the break even if you will right it's okay if a few people
6:05
you will right it's okay if a few people
6:05
you will right it's okay if a few people are not using it at all but no it's not
6:08
are not using it at all but no it's not
6:08
are not using it at all but no it's not that's a waste ofense and that's that's
6:09
that's a waste ofense and that's that's
6:09
that's a waste ofense and that's that's the question right yeah yeah so the what
6:12
the question right yeah yeah so the what
6:12
the question right yeah yeah so the what people are doing now is that they're
6:13
people are doing now is that they're
6:13
people are doing now is that they're saying after a
6:15
saying after a year who's using it who's not and I've
6:18
year who's using it who's not and I've
6:18
year who's using it who's not and I've got this pool of co-pilot licenses how
6:21
got this pool of co-pilot licenses how
6:21
got this pool of co-pilot licenses how can I make sure that I can give those
6:22
can I make sure that I can give those
6:22
can I make sure that I can give those co-pilot licenses the people who
6:24
co-pilot licenses the people who
6:24
co-pilot licenses the people who actually use it yes because if you don't
6:27
actually use it yes because if you don't
6:27
actually use it yes because if you don't use a technology you get zero benefit
6:29
use a technology you get zero benefit
6:29
use a technology you get zero benefit from us why are we paying for it right
6:31
from us why are we paying for it right
6:31
from us why are we paying for it right exactly that the problem that the den
6:33
exactly that the problem that the den
6:33
exactly that the problem that the den come up with is to
6:35
come up with is to find that the information which is
6:37
find that the information which is
6:37
find that the information which is available let's say in the Microsoft 365
6:40
available let's say in the Microsoft 365
6:40
available let's say in the Microsoft 365 admin Center yes there's a tell you
6:42
admin Center yes there's a tell you
6:42
admin Center yes there's a tell you usage report for
6:44
usage report for co-pilot
6:45
co-pilot is very very skimpy I mean if you use so
6:51
is very very skimpy I mean if you use so
6:51
is very very skimpy I mean if you use so here's an example yeah word yeah if you
6:54
here's an example yeah word yeah if you
6:54
here's an example yeah word yeah if you got a co-pilot license world will
6:57
got a co-pilot license world will
6:57
got a co-pilot license world will automatically generate things summary of
7:00
automatically generate things summary of
7:00
automatically generate things summary of a document when you open it that's
7:02
a document when you open it that's
7:02
a document when you open it that's pretty good
7:03
pretty good yeah but to Microsoft to the usage
7:06
yeah but to Microsoft to the usage
7:06
yeah but to Microsoft to the usage report data that Microsoft accumulates
7:09
report data that Microsoft accumulates
7:09
report data that Microsoft accumulates that indicates that you're a very heavy
7:10
that indicates that you're a very heavy
7:10
that indicates that you're a very heavy user of word co-pilot and word I would
7:13
user of word co-pilot and word I would
7:13
user of word co-pilot and word I would say you're absolutely not using co-pilot
7:15
say you're absolutely not using co-pilot
7:15
say you're absolutely not using co-pilot word because this has been done
7:17
word because this has been done
7:17
word because this has been done automatically for what do you get any
7:18
automatically for what do you get any
7:19
automatically for what do you get any benefit unless you actually use the data
7:21
benefit unless you actually use the data
7:21
benefit unless you actually use the data that turns into sort of a false
7:24
that turns into sort of a false
7:24
that turns into sort of a false positive well no it's just Microsoft is
7:26
positive well no it's just Microsoft is
7:26
positive well no it's just Microsoft is pumping up the amount of of usage of it
7:28
pumping up the amount of of usage of it
7:28
pumping up the amount of of usage of it and there the people will look at and
7:30
and there the people will look at and
7:30
and there the people will look at and say oh the summaries are useful
7:32
say oh the summaries are useful
7:32
say oh the summaries are useful therefore I can use copilot to do other
7:34
therefore I can use copilot to do other
7:34
therefore I can use copilot to do other stuff and I can see why they're doing it
7:36
stuff and I can see why they're doing it
7:36
stuff and I can see why they're doing it okay but the point is that if you're
7:38
okay but the point is that if you're
7:38
okay but the point is that if you're looking at the Hard data to measure
7:40
looking at the Hard data to measure
7:40
looking at the Hard data to measure against and you're only going to you're
7:43
against and you're only going to you're
7:43
against and you're only going to you're seeing false readings that's what I'm
7:45
seeing false readings that's what I'm
7:45
seeing false readings that's what I'm saying it's looks like there's a lot of
7:47
saying it's looks like there's a lot of
7:47
saying it's looks like there's a lot of people using those summaries but they
7:49
people using those summaries but they
7:49
people using those summaries but they are in fact just automatically there
7:51
are in fact just automatically there
7:51
are in fact just automatically there well you don't even no you don't even
7:53
well you don't even no you don't even
7:53
well you don't even no you don't even know that use know that you're using
7:55
know that use know that you're using
7:55
know that use know that you're using summaries right right right you know
7:57
summaries right right right you know
7:57
summaries right right right you know that they're using Word copon or cop
8:00
that they're using Word copon or cop
8:00
that they're using Word copon or cop words that's right it appears like but
8:02
words that's right it appears like but
8:02
words that's right it appears like but there are other things I mean it doesn't
8:05
there are other things I mean it doesn't
8:05
there are other things I mean it doesn't mean that they're using copilot and word
8:08
mean that they're using copilot and word
8:08
mean that they're using copilot and word to generate text it doesn't mean that
8:10
to generate text it doesn't mean that
8:10
to generate text it doesn't mean that they're using copilot and word to refine
8:12
they're using copilot and word to refine
8:12
they're using copilot and word to refine text that's right both of which I would
8:14
text that's right both of which I would
8:14
text that's right both of which I would hold to be more high value activities
8:17
hold to be more high value activities
8:17
hold to be more high value activities than a summary true yeah right
8:19
than a summary true yeah right
8:19
than a summary true yeah right especially an automatic summary that's
8:21
especially an automatic summary that's
8:21
especially an automatic summary that's derived from a document that I never
8:23
derived from a document that I never
8:23
derived from a document that I never asked for to so you're telling me that
8:25
asked for to so you're telling me that
8:25
asked for to so you're telling me that you can read the the the the logs and
8:28
you can read the the the the logs and
8:28
you can read the the the the logs and see somebody is using word
8:31
see somebody is using word
8:31
see somebody is using word co-pilot that's basically because they
8:33
co-pilot that's basically because they
8:33
co-pilot that's basically because they open the document in Word but you can't
8:35
open the document in Word but you can't
8:35
open the document in Word but you can't differentiate what they're actually
8:37
differentiate what they're actually
8:37
differentiate what they're actually doing well what I'm saying no no what
8:39
doing well what I'm saying no no what
8:39
doing well what I'm saying no no what I'm saying is that the usage so yeah in
8:42
I'm saying is that the usage so yeah in
8:42
I'm saying is that the usage so yeah in the Microsoft 365 admin Center you have
8:45
the Microsoft 365 admin Center you have
8:45
the Microsoft 365 admin Center you have usage reports yeah user reports are
8:47
usage reports yeah user reports are
8:47
usage reports yeah user reports are based out of a data warehouse yeah and
8:51
based out of a data warehouse yeah and
8:51
based out of a data warehouse yeah and are accessed through a graph API yes the
8:53
are accessed through a graph API yes the
8:53
are accessed through a graph API yes the graph API is not very granular no and
8:57
graph API is not very granular no and
8:57
graph API is not very granular no and what I'm saying is that when it reports
8:59
what I'm saying is that when it reports
8:59
what I'm saying is that when it reports to to you yeah that this user is a heavy
9:02
to to you yeah that this user is a heavy
9:02
to to you yeah that this user is a heavy consumer of copil and word you have no
9:06
consumer of copil and word you have no
9:06
consumer of copil and word you have no idea whether that
9:07
idea whether that user actually is just seing all these
9:10
user actually is just seing all these
9:10
user actually is just seing all these automated summaries are they actually
9:12
automated summaries are they actually
9:12
automated summaries are they actually interacting with copilot and word in
9:14
interacting with copilot and word in
9:14
interacting with copilot and word in some productive manner doing something
9:16
some productive manner doing something
9:16
some productive manner doing something else actively with it right so the usage
9:19
else actively with it right so the usage
9:19
else actively with it right so the usage data is not very good and what I've
9:21
data is not very good and what I've
9:21
data is not very good and what I've heard from a lot of customers is that
9:24
heard from a lot of customers is that
9:24
heard from a lot of customers is that they don't have enough granular
9:26
they don't have enough granular
9:26
they don't have enough granular information to figure out if who is
9:31
information to figure out if who is
9:31
information to figure out if who is actually
9:33
actually Ling to to use the AI cuz you know a
9:36
Ling to to use the AI cuz you know a
9:36
Ling to to use the AI cuz you know a summary is not really a very good use of
9:38
summary is not really a very good use of
9:38
summary is not really a very good use of of Ai No not in a document not in a
9:41
of Ai No not in a document not in a
9:41
of Ai No not in a document not in a docum it is in something like if you to
9:43
docum it is in something like if you to
9:43
docum it is in something like if you to summarize an email thread or to
9:45
summarize an email thread or to
9:45
summarize an email thread or to summarize a teams meeting teams meeting
9:48
summarize a teams meeting teams meeting
9:48
summarize a teams meeting teams meeting is good absolutely summary fine but to
9:51
is good absolutely summary fine but to
9:51
is good absolutely summary fine but to have this automatic creation of uh of a
9:54
have this automatic creation of uh of a
9:54
have this automatic creation of uh of a document summary I'm sorry that's not a
9:55
document summary I'm sorry that's not a
9:56
document summary I'm sorry that's not a good use of Ai No it's not necessarily a
9:57
good use of Ai No it's not necessarily a
9:57
good use of Ai No it's not necessarily a good use of AI unless you explicitly
10:01
good use of AI unless you explicitly
10:01
good use of AI unless you explicitly have you have a document and I'm I'm I'm
10:03
have you have a document and I'm I'm I'm
10:03
have you have a document and I'm I'm I'm really going to like this this is so
10:05
really going to like this this is so
10:05
really going to like this this is so much information I'm going to actively
10:08
much information I'm going to actively
10:08
much information I'm going to actively do it maybe co-pilot can't deal with
10:10
do it maybe co-pilot can't deal with
10:10
do it maybe co-pilot can't deal with very large documents so there's that
10:13
very large documents so there's that
10:13
very large documents so there's that there's that so what can we do in this
10:15
there's that so what can we do in this
10:15
there's that so what can we do in this space to uh
10:19
space to uh help company leaders assess the value of
10:23
help company leaders assess the value of
10:23
help company leaders assess the value of R return on investment on co-pilot
10:24
R return on investment on co-pilot
10:24
R return on investment on co-pilot licenses well I think it all comes down
10:26
licenses well I think it all comes down
10:26
licenses well I think it all comes down to data I mean Microsoft is famed for
10:28
to data I mean Microsoft is famed for
10:28
to data I mean Microsoft is famed for talking about their Telemetry yeah uh
10:31
talking about their Telemetry yeah uh
10:31
talking about their Telemetry yeah uh they they they say that the decisions
10:33
they they they say that the decisions
10:33
they they they say that the decisions they make about where they're going to
10:34
they make about where they're going to
10:34
they make about where they're going to invest in is through Telemetry they look
10:36
invest in is through Telemetry they look
10:36
invest in is through Telemetry they look at stuff and they F what features are
10:38
at stuff and they F what features are
10:38
at stuff and they F what features are users are people using and they're the
10:41
users are people using and they're the
10:41
users are people using and they're the ones they're going to invest in so I
10:42
ones they're going to invest in so I
10:42
ones they're going to invest in so I turn that around and say well what data
10:44
turn that around and say well what data
10:44
turn that around and say well what data is available to you that's right and the
10:46
is available to you that's right and the
10:46
is available to you that's right and the data that's available to you is in the
10:48
data that's available to you is in the
10:48
data that's available to you is in the audit log because you can parse the
10:50
audit log because you can parse the
10:50
audit log because you can parse the audit log data in fact there was a great
10:51
audit log data in fact there was a great
10:51
audit log data in fact there was a great session here yesterday by a guy from
10:54
session here yesterday by a guy from
10:54
session here yesterday by a guy from Finland whose Name Escapes Me totally
10:56
Finland whose Name Escapes Me totally
10:56
Finland whose Name Escapes Me totally but he talked about how using some of
10:58
but he talked about how using some of
10:58
but he talked about how using some of the a log data he was able to take that
11:01
the a log data he was able to take that
11:01
the a log data he was able to take that par it out and then push it into push it
11:03
par it out and then push it into push it
11:03
par it out and then push it into push it into powerbi and I think the name you're
11:06
into powerbi and I think the name you're
11:06
into powerbi and I think the name you're looking for is sakar nahi could be yeah
11:09
looking for is sakar nahi could be yeah
11:09
looking for is sakar nahi could be yeah anyway could be yeah I mean like it
11:10
anyway could be yeah I mean like it
11:10
anyway could be yeah I mean like it sounds a good name think so yeah yeah
11:12
sounds a good name think so yeah yeah
11:12
sounds a good name think so yeah yeah I'd have to look it I you know I was
11:14
I'd have to look it I you know I was
11:14
I'd have to look it I you know I was just sitting in the the audience in fact
11:16
just sitting in the the audience in fact
11:16
just sitting in the the audience in fact i' I've been um I have a version of a
11:20
i' I've been um I have a version of a
11:20
i' I've been um I have a version of a par shell script that that's written but
11:23
par shell script that that's written but
11:23
par shell script that that's written but I haven't actually published the article
11:24
I haven't actually published the article
11:24
I haven't actually published the article yet yeah which can takes data from the
11:28
yet yeah which can takes data from the
11:28
yet yeah which can takes data from the usage logs and scores People based on
11:31
usage logs and scores People based on
11:31
usage logs and scores People based on what we what if you like fingerprints we
11:33
what we what if you like fingerprints we
11:33
what we what if you like fingerprints we find and using copilot words and teams
11:35
find and using copilot words and teams
11:35
find and using copilot words and teams and Excel and PowerPoint in Loop and one
11:38
and Excel and PowerPoint in Loop and one
11:38
and Excel and PowerPoint in Loop and one drive one note and all the rest of it
11:41
drive one note and all the rest of it
11:41
drive one note and all the rest of it but then also says okay that's the basic
11:43
but then also says okay that's the basic
11:43
but then also says okay that's the basic data uh that's a non-granular data
11:46
data uh that's a non-granular data
11:46
data uh that's a non-granular data that's just a marker that you have
11:48
that's just a marker that you have
11:48
that's just a marker that you have actually interacted with co-pilot okay
11:50
actually interacted with co-pilot okay
11:50
actually interacted with co-pilot okay in these workloads at least once a day
11:52
in these workloads at least once a day
11:52
in these workloads at least once a day to set the flag okay but then associated
11:56
to set the flag okay but then associated
11:56
to set the flag okay but then associated with that for each user we have AED log
11:59
with that for each user we have AED log
12:00
with that for each user we have AED log data which says okay when we de dive
12:04
data which says okay when we de dive
12:04
data which says okay when we de dive deeper we can see that there were these
12:06
deeper we can see that there were these
12:06
deeper we can see that there were these kind of
12:07
kind of interactions now if you want to dive
12:10
interactions now if you want to dive
12:10
interactions now if you want to dive really deep and this is where we get
12:12
really deep and this is where we get
12:12
really deep and this is where we get forensically yeah uh deep what you can
12:14
forensically yeah uh deep what you can
12:14
forensically yeah uh deep what you can use is the graph apis to open an
12:19
use is the graph apis to open an
12:19
use is the graph apis to open an exchange mailbox the user's mailbox mhm
12:22
exchange mailbox the user's mailbox mhm
12:22
exchange mailbox the user's mailbox mhm and go to the hidden folder where the
12:24
and go to the hidden folder where the
12:24
and go to the hidden folder where the compliance records wow are
12:27
compliance records wow are
12:27
compliance records wow are kept by paring the details of the
12:29
kept by paring the details of the
12:29
kept by paring the details of the compliance records you can find the
12:31
compliance records you can find the
12:32
compliance records you can find the prompts and the responses oh what they
12:34
prompts and the responses oh what they
12:34
prompts and the responses oh what they actually yeah were asking and and the
12:37
actually yeah were asking and and the
12:37
actually yeah were asking and and the responses to exactly now that is
12:40
responses to exactly now that is
12:40
responses to exactly now that is uh
12:44
that's a huge I it's an interesting exercise in
12:47
huge I it's an interesting exercise in
12:47
huge I it's an interesting exercise in computer science oh yeah right yeah yeah
12:49
computer science oh yeah right yeah yeah
12:49
computer science oh yeah right yeah yeah but it's not something I necessarily
12:51
but it's not something I necessarily
12:51
but it's not something I necessarily recommend unless you're in the middle of
12:52
recommend unless you're in the middle of
12:52
recommend unless you're in the middle of a forensic investigation and you have
12:55
a forensic investigation and you have
12:55
a forensic investigation and you have got the sign off which is necessary to
12:58
got the sign off which is necessary to
12:58
got the sign off which is necessary to approve invading somebody's personal
13:01
approve invading somebody's personal
13:01
approve invading somebody's personal mailbox that's right to have a look
13:03
mailbox that's right to have a look
13:03
mailbox that's right to have a look because you know some of those prompts
13:05
because you know some of those prompts
13:05
because you know some of those prompts and some of those responses may contain
13:07
and some of those responses may contain
13:07
and some of those responses may contain pii yeah yeah yeah yeah okay so it's not
13:10
pii yeah yeah yeah yeah okay so it's not
13:10
pii yeah yeah yeah yeah okay so it's not an exercise that I would suggest just in
13:13
an exercise that I would suggest just in
13:13
an exercise that I would suggest just in general people doing you know and I
13:14
general people doing you know and I
13:14
general people doing you know and I don't suggest that any administrator for
13:16
don't suggest that any administrator for
13:16
don't suggest that any administrator for example takes the code that we Ro using
13:19
example takes the code that we Ro using
13:19
example takes the code that we Ro using the graph to do this makes sense to go
13:21
the graph to do this makes sense to go
13:21
the graph to do this makes sense to go and say well well let me take the um the
13:26
and say well well let me take the um the
13:26
and say well well let me take the um the mailboxes of the CEO and their direct
13:29
mailboxes of the CEO and their direct
13:29
mailboxes of the CEO and their direct reports and I'm going to just going to
13:31
reports and I'm going to just going to
13:31
reports and I'm going to just going to poke through their co-pilot records and
13:33
poke through their co-pilot records and
13:33
poke through their co-pilot records and find out what they see I can see some
13:35
find out what they see I can see some
13:35
find out what they see I can see some issues there the kind of privacy there
13:37
issues there the kind of privacy there
13:37
issues there the kind of privacy there but the point is is that the data is
13:39
but the point is is that the data is
13:39
but the point is is that the data is there if you want to use the data you
13:41
there if you want to use the data you
13:41
there if you want to use the data you want to extract the data y but you got
13:43
want to extract the data y but you got
13:43
want to extract the data y but you got to have a reason to use it and the
13:45
to have a reason to use it and the
13:45
to have a reason to use it and the reason why that I'm seeing people coming
13:47
reason why that I'm seeing people coming
13:47
reason why that I'm seeing people coming to now is as I say it's about a year
13:50
to now is as I say it's about a year
13:50
to now is as I say it's about a year they're now reviewing and saying okay
13:53
they're now reviewing and saying okay
13:53
they're now reviewing and saying okay maybe I bought 600 co-pilot licenses do
13:55
maybe I bought 600 co-pilot licenses do
13:55
maybe I bought 600 co-pilot licenses do I need 800 yeah or do I need 400 or do
13:58
I need 800 yeah or do I need 400 or do
13:58
I need 800 yeah or do I need 400 or do we need 400 and if I only need 400 who
14:00
we need 400 and if I only need 400 who
14:01
we need 400 and if I only need 400 who am I going to give them to yeah and one
14:03
am I going to give them to yeah and one
14:03
am I going to give them to yeah and one I've I've talked to a couple of
14:04
I've I've talked to a couple of
14:04
I've I've talked to a couple of companies who saying well what we
14:06
companies who saying well what we
14:06
companies who saying well what we need is something like an aour
14:09
need is something like an aour
14:09
need is something like an aour automation job that runs once a month in
14:12
automation job that runs once a month in
14:12
automation job that runs once a month in the background analyzes all this data
14:15
the background analyzes all this data
14:15
the background analyzes all this data mhm and let's say it says well you're
14:18
mhm and let's say it says well you're
14:18
mhm and let's say it says well you're not using co-pilot very much but I know
14:21
not using co-pilot very much but I know
14:21
not using co-pilot very much but I know you're on vacation so that's okay uh
14:23
you're on vacation so that's okay uh
14:23
you're on vacation so that's okay uh okay but this other chap over here he's
14:26
okay but this other chap over here he's
14:26
okay but this other chap over here he's not using copilot very much so I'm going
14:28
not using copilot very much so I'm going
14:28
not using copilot very much so I'm going to take C
14:29
to take C away from that person and I'm going to
14:31
away from that person and I'm going to
14:31
away from that person and I'm going to give it to a candidate user and a
14:34
give it to a candidate user and a
14:34
give it to a candidate user and a candidate user might be somebody who
14:37
candidate user might be somebody who
14:37
candidate user might be somebody who who's in a department where we know has
14:40
who's in a department where we know has
14:40
who's in a department where we know has heavy co-pilot usage but they don't have
14:42
heavy co-pilot usage but they don't have
14:42
heavy co-pilot usage but they don't have it and they're at at a particular level
14:44
it and they're at at a particular level
14:44
it and they're at at a particular level there are lots of ways I mean this is
14:45
there are lots of ways I mean this is
14:45
there are lots of ways I mean this is just data manipulation but they're
14:47
just data manipulation but they're
14:47
just data manipulation but they're talking about doing that kind of thing
14:50
talking about doing that kind of thing
14:50
talking about doing that kind of thing so you have a dynamic shift of co-pilot
14:53
so you have a dynamic shift of co-pilot
14:53
so you have a dynamic shift of co-pilot licenses but you need to drive it with
14:55
licenses but you need to drive it with
14:55
licenses but you need to drive it with some decision decision data otherwise
14:58
some decision decision data otherwise
14:58
some decision decision data otherwise you can't know who would get or not
15:00
you can't know who would get or not
15:00
you can't know who would get or not that's the kind of stuff we're we're
15:02
that's the kind of stuff we're we're
15:02
that's the kind of stuff we're we're working on interesting well that's a
15:04
working on interesting well that's a
15:04
working on interesting well that's a very interesting conversation indeed but
15:06
very interesting conversation indeed but
15:06
very interesting conversation indeed but that's all the time we have for this
15:08
that's all the time we have for this
15:08
that's all the time we have for this episode I want to thank you for coming
15:10
episode I want to thank you for coming
15:10
episode I want to thank you for coming on the cloud show to talk to me about
15:11
on the cloud show to talk to me about
15:11
on the cloud show to talk to me about this thank you very much Tony Che and
15:14
this thank you very much Tony Che and
15:14
this thank you very much Tony Che and audience we'll see you guys next time on
15:15
audience we'll see you guys next time on
15:15
audience we'll see you guys next time on the cloud show