Up next in 10
Simplifying Multi-Cloud Observability | Software Architecture Conference
Aug 6, 2025
Enterprises managing multi-cloud environments face significant challenges in achieving comprehensive observability across diverse infrastructures. This talk highlights the essential role of standardized, vendor-agnostic approaches in ensuring scalability and usability in modern monitoring practices. We'll explore the complexities of observability, focusing on the need for consistent management of metrics and logs across multiple cloud providers. A detailed case study will demonstrate how open-source tools like OpenTelemetry can unify data collection without vendor lock-in. Attendees will gain actionable insights to enhance visibility and operational resilience in their multi-cloud deployments. 🔗 Conference Website: https://softwarearchitecture.live 📺 CSharp TV - Dev Streaming Destination http://csharp.tv 🌎 C# Corner - Community of Software and Data Developers https://www.c-sharpcorner.com #CSharpTV #CSharpCorner #CSharp #SoftwareArchitectureConf
View Video Transcript
0:03
hello everyone my name is sirma vaju
0:06
hello everyone my name is sirma vaju
0:06
hello everyone my name is sirma vaju first of all I would like to thank C
0:09
first of all I would like to thank C
0:09
first of all I would like to thank C Corner for giving me this opportunity to
0:11
Corner for giving me this opportunity to
0:11
Corner for giving me this opportunity to present my talk on this topic
0:14
present my talk on this topic
0:14
present my talk on this topic simplifying multicloud
0:16
simplifying multicloud
0:16
simplifying multicloud observability in this session we'll talk
0:20
observability in this session we'll talk
0:20
observability in this session we'll talk about multicloud
0:22
about multicloud
0:22
about multicloud observability what are the complexities
0:25
observability what are the complexities
0:25
observability what are the complexities of observability in a multic cloud
0:26
of observability in a multic cloud
0:26
of observability in a multic cloud architecture and how to simplify it
0:31
architecture and how to simplify it
0:31
architecture and how to simplify it before we de Deep dive a little bit
0:33
before we de Deep dive a little bit
0:33
before we de Deep dive a little bit about myself I am sirma vei Raju a
0:36
about myself I am sirma vei Raju a
0:36
about myself I am sirma vei Raju a software engineer at Microsoft previous
0:38
software engineer at Microsoft previous
0:38
software engineer at Microsoft previous year tacle in my free time I love to
0:41
year tacle in my free time I love to
0:41
year tacle in my free time I love to contribute to open source and I'm an
0:43
contribute to open source and I'm an
0:43
contribute to open source and I'm an evid hiker currently I live in Seattle
0:46
evid hiker currently I live in Seattle
0:46
evid hiker currently I live in Seattle and Seattle summers are the best and a
0:49
and Seattle summers are the best and a
0:49
and Seattle summers are the best and a hike on a weekend is what makes it more
0:52
hike on a weekend is what makes it more
0:52
hike on a weekend is what makes it more amazing I also love to review books and
0:56
amazing I also love to review books and
0:56
amazing I also love to review books and this is my LinkedIn email and Twitter
0:59
this is my LinkedIn email and Twitter
0:59
this is my LinkedIn email and Twitter please please feel free to reach out to
1:01
please please feel free to reach out to
1:01
please please feel free to reach out to me if you have any feedback or if you
1:04
me if you have any feedback or if you
1:04
me if you have any feedback or if you want to talk anything about Cloud
1:07
want to talk anything about Cloud
1:07
want to talk anything about Cloud observability software engineering and
1:09
observability software engineering and
1:09
observability software engineering and distributed systems in general I'm
1:12
distributed systems in general I'm
1:12
distributed systems in general I'm always open to talking to folks and
1:13
always open to talking to folks and
1:13
always open to talking to folks and learn from their experience with that
1:16
learn from their experience with that
1:16
learn from their experience with that let's get started with the
1:19
let's get started with the
1:19
let's get started with the talk so looking at the agenda first
1:22
talk so looking at the agenda first
1:22
talk so looking at the agenda first we'll start with what is multi Cloud why
1:24
we'll start with what is multi Cloud why
1:24
we'll start with what is multi Cloud why is it gaining a lot of popularity and
1:27
is it gaining a lot of popularity and
1:27
is it gaining a lot of popularity and what are the motivations behind
1:29
what are the motivations behind
1:29
what are the motivations behind Enterprises is moving towards this
1:31
Enterprises is moving towards this
1:31
Enterprises is moving towards this architecture next we look at
1:34
architecture next we look at
1:34
architecture next we look at observability what is observability how
1:36
observability what is observability how
1:36
observability what is observability how does observability help you track your
1:38
does observability help you track your
1:39
does observability help you track your service health and what are the three
1:41
service health and what are the three
1:41
service health and what are the three main Concepts in
1:43
main Concepts in
1:43
main Concepts in observability then we'll take a look at
1:47
observability then we'll take a look at
1:47
observability then we'll take a look at why the
1:48
why the
1:48
why the observability approach needs to change
1:51
observability approach needs to change
1:51
observability approach needs to change between a single and multicloud
1:53
between a single and multicloud
1:53
between a single and multicloud environment what are the things that
1:56
environment what are the things that
1:56
environment what are the things that demand a change in approach when
2:00
demand a change in approach when
2:00
demand a change in approach when considering the multicloud
2:02
considering the multicloud
2:02
considering the multicloud architecture then we'll take a look at
2:04
architecture then we'll take a look at
2:05
architecture then we'll take a look at the complexities and how to simplify
2:07
the complexities and how to simplify
2:07
the complexities and how to simplify observability and at the last we'll
2:09
observability and at the last we'll
2:09
observability and at the last we'll conclude the
2:13
talk so what is multic cloud and why is
2:16
talk so what is multic cloud and why is
2:16
talk so what is multic cloud and why is it gaining a lot of
2:17
it gaining a lot of
2:17
it gaining a lot of popularity you see when Cloud was just a
2:20
popularity you see when Cloud was just a
2:20
popularity you see when Cloud was just a buzz word people or companies hosted
2:23
buzz word people or companies hosted
2:23
buzz word people or companies hosted most of the services on
2:26
most of the services on
2:26
most of the services on PR when Cloud adoption started
2:30
PR when Cloud adoption started
2:30
PR when Cloud adoption started companies started hosting some of their
2:32
companies started hosting some of their
2:32
companies started hosting some of their workloads on on cloud and on PR that is
2:36
workloads on on cloud and on PR that is
2:36
workloads on on cloud and on PR that is when they called it hybri hybrid cloud
2:39
when they called it hybri hybrid cloud
2:39
when they called it hybri hybrid cloud and then as the cloud adoption became
2:42
and then as the cloud adoption became
2:42
and then as the cloud adoption became stronger and as sophistications in how
2:45
stronger and as sophistications in how
2:45
stronger and as sophistications in how we built software grew
2:48
we built software grew
2:48
we built software grew and building data centers is also not
2:51
and building data centers is also not
2:51
and building data centers is also not cheap so companies started completely
2:55
cheap so companies started completely
2:55
cheap so companies started completely using cloud and when I as a company use
2:59
using cloud and when I as a company use
2:59
using cloud and when I as a company use more than than one cloud provider to
3:02
more than than one cloud provider to
3:02
more than than one cloud provider to serve my software that is when I call it
3:04
serve my software that is when I call it
3:05
serve my software that is when I call it as a multicloud
3:07
as a multicloud
3:07
as a multicloud architecture and a staggering 98% this
3:10
architecture and a staggering 98% this
3:10
architecture and a staggering 98% this is part of a recent Oracle survey where
3:14
is part of a recent Oracle survey where
3:14
is part of a recent Oracle survey where what they have seen as 98% of
3:18
what they have seen as 98% of
3:18
what they have seen as 98% of Enterprises 98% of Enterprise companies
3:22
Enterprises 98% of Enterprise companies
3:22
Enterprises 98% of Enterprise companies are either thinking about the multicloud
3:24
are either thinking about the multicloud
3:24
are either thinking about the multicloud strategy or are already on the
3:27
strategy or are already on the
3:27
strategy or are already on the multicloud strategy and
3:30
multicloud strategy and
3:30
multicloud strategy and the key motivations for these companies
3:33
the key motivations for these companies
3:33
the key motivations for these companies to move towards this approach is First
3:36
to move towards this approach is First
3:36
to move towards this approach is First Data
3:38
Data
3:38
Data sovereignty so considered I'm an
3:40
sovereignty so considered I'm an
3:40
sovereignty so considered I'm an Enterprise company I have my customers
3:44
Enterprise company I have my customers
3:44
Enterprise company I have my customers worldwide I have them in different
3:46
worldwide I have them in different
3:46
worldwide I have them in different continents geopolitical regions and
3:48
continents geopolitical regions and
3:48
continents geopolitical regions and different countries and each country has
3:51
different countries and each country has
3:51
different countries and each country has their own
3:53
their own
3:53
their own gdpr has their own data privacy and
3:55
gdpr has their own data privacy and
3:55
gdpr has their own data privacy and protection laws for example United
3:58
protection laws for example United
3:58
protection laws for example United States has its own data privacy data
4:00
States has its own data privacy data
4:00
States has its own data privacy data Protection
4:01
Protection
4:01
Protection Law European Union has gdpr similarly
4:05
Law European Union has gdpr similarly
4:05
Law European Union has gdpr similarly India China Japan Australia all these
4:08
India China Japan Australia all these
4:08
India China Japan Australia all these countries have their own laws and if I
4:10
countries have their own laws and if I
4:10
countries have their own laws and if I as an Enterprise company understand that
4:13
as an Enterprise company understand that
4:13
as an Enterprise company understand that my current cloud provider does
4:16
my current cloud provider does
4:16
my current cloud provider does not allow me does not allow me to or
4:21
not allow me does not allow me to or
4:21
not allow me does not allow me to or does
4:24
not does not allow me to use some of
4:27
not does not allow me to use some of
4:27
not does not allow me to use some of this capabilities that is when I would
4:30
this capabilities that is when I would
4:30
this capabilities that is when I would choose another cloud provider
4:33
choose another cloud provider
4:33
choose another cloud provider basically where I get to is the cloud
4:36
basically where I get to is the cloud
4:36
basically where I get to is the cloud provider does not meet my needs so I
4:38
provider does not meet my needs so I
4:38
provider does not meet my needs so I have to go looking out for another cloud
4:42
have to go looking out for another cloud
4:42
have to go looking out for another cloud provider that is one reason the second
4:44
provider that is one reason the second
4:44
provider that is one reason the second reason is vendor
4:47
reason is vendor
4:47
reason is vendor agnostic if I as a company depend
4:51
agnostic if I as a company depend
4:51
agnostic if I as a company depend on depend on a service that cloud
4:54
on depend on a service that cloud
4:54
on depend on a service that cloud provider offers and if for some reason
4:57
provider offers and if for some reason
4:57
provider offers and if for some reason either the service is being deprecated
4:59
either the service is being deprecated
4:59
either the service is being deprecated ated or the cost is being increased by
5:03
ated or the cost is being increased by
5:04
ated or the cost is being increased by the
5:05
the
5:05
the provider or the features drastically
5:07
provider or the features drastically
5:07
provider or the features drastically change I as a company want to be
5:10
change I as a company want to be
5:10
change I as a company want to be agnostic of
5:12
agnostic of
5:12
agnostic of this I don't want my business deadlines
5:16
this I don't want my business deadlines
5:16
this I don't want my business deadlines to be affected because of these changes
5:19
to be affected because of these changes
5:19
to be affected because of these changes by the provider and that is one more
5:21
by the provider and that is one more
5:21
by the provider and that is one more reason why I
5:23
reason why I
5:23
reason why I would be vendor agnostic and that is
5:26
would be vendor agnostic and that is
5:26
would be vendor agnostic and that is when I would choose a multicloud
5:28
when I would choose a multicloud
5:28
when I would choose a multicloud architecture third cost
5:31
architecture third cost
5:31
architecture third cost optimizations if I think about
5:34
optimizations if I think about
5:34
optimizations if I think about it every cloud provider has their own
5:37
it every cloud provider has their own
5:37
it every cloud provider has their own strong points some offer free gr and
5:39
strong points some offer free gr and
5:39
strong points some offer free gr and Ingress some offer cheap storage and
5:41
Ingress some offer cheap storage and
5:41
Ingress some offer cheap storage and some offer free VMS or cheap
5:44
some offer free VMS or cheap
5:44
some offer free VMS or cheap VMS now if I as a company am in a place
5:49
VMS now if I as a company am in a place
5:49
VMS now if I as a company am in a place to use
5:51
to use
5:52
to use these to use these features then I can
5:56
these to use these features then I can
5:56
these to use these features then I can significantly reduce the cost
5:59
significantly reduce the cost
5:59
significantly reduce the cost it takes to run my services on cloud and
6:02
it takes to run my services on cloud and
6:02
it takes to run my services on cloud and we all know that running services in
6:05
we all know that running services in
6:05
we all know that running services in cloud is pretty expensive so keeping a
6:08
cloud is pretty expensive so keeping a
6:08
cloud is pretty expensive so keeping a vendor agnostic architecture will help
6:11
vendor agnostic architecture will help
6:11
vendor agnostic architecture will help me leverage these features and scale
6:18
me leverage these features and scale
6:18
me leverage these features and scale better to conclude the three main
6:20
better to conclude the three main
6:21
better to conclude the three main motivations being data sovereignty Cloud
6:23
motivations being data sovereignty Cloud
6:23
motivations being data sovereignty Cloud vend Diagnostic and cost optimizations
6:26
vend Diagnostic and cost optimizations
6:27
vend Diagnostic and cost optimizations of course there are other things as well
6:28
of course there are other things as well
6:28
of course there are other things as well but these are the three main
6:33
contributions next we'll take a look at
6:36
contributions next we'll take a look at
6:36
contributions next we'll take a look at observability so what is
6:39
observability so what is
6:39
observability so what is observability simply put the ability to
6:42
observability simply put the ability to
6:42
observability simply put the ability to measure the current state of my
6:45
measure the current state of my
6:45
measure the current state of my system is my system healthy is my system
6:49
system is my system healthy is my system
6:49
system is my system healthy is my system the request latency is low is my am I
6:52
the request latency is low is my am I
6:52
the request latency is low is my am I meeting my service level SLI slos and
6:57
meeting my service level SLI slos and
6:57
meeting my service level SLI slos and slas all these answers I can get from
7:01
slas all these answers I can get from
7:01
slas all these answers I can get from observability the better the
7:03
observability the better the
7:03
observability the better the observability story for your company or
7:05
observability story for your company or
7:05
observability story for your company or for your product the better you're able
7:08
for your product the better you're able
7:08
for your product the better you're able to serve your
7:09
to serve your
7:10
to serve your customers
7:11
customers
7:11
customers and there are three main Concepts in
7:14
and there are three main Concepts in
7:14
and there are three main Concepts in observability metrics logs and traces
7:18
observability metrics logs and traces
7:18
observability metrics logs and traces let's look at each of
7:21
let's look at each of
7:21
let's look at each of them so coming to metrics metrics are
7:25
them so coming to metrics metrics are
7:25
them so coming to metrics metrics are the numerical representations of your
7:27
the numerical representations of your
7:27
the numerical representations of your system perform system performance for
7:29
system perform system performance for
7:29
system perform system performance for example let's say I am serving a request
7:33
example let's say I am serving a request
7:33
example let's say I am serving a request and I emit a one for a success and zero
7:36
and I emit a one for a success and zero
7:36
and I emit a one for a success and zero for a failure now I'm able
7:39
for a failure now I'm able
7:39
for a failure now I'm able to plot a mean of this graph which gives
7:43
to plot a mean of this graph which gives
7:43
to plot a mean of this graph which gives me your five9 or four nines of
7:47
me your five9 or four nines of
7:47
me your five9 or four nines of availability and if I as a company
7:51
availability and if I as a company
7:51
availability and if I as a company promise three above Three N or four NES
7:55
promise three above Three N or four NES
7:55
promise three above Three N or four NES of
7:56
of
7:56
of availability these metrics help me
7:58
availability these metrics help me
7:58
availability these metrics help me understand if I'm able to meet that and
8:01
understand if I'm able to meet that and
8:01
understand if I'm able to meet that and if
8:02
if
8:02
if not I monitor I write alert so that I
8:08
not I monitor I write alert so that I
8:08
not I monitor I write alert so that I understand the system degradation and
8:10
understand the system degradation and
8:10
understand the system degradation and can mitigate the problem as soon as
8:13
can mitigate the problem as soon as
8:13
can mitigate the problem as soon as possible and coming to metrics there are
8:15
possible and coming to metrics there are
8:15
possible and coming to metrics there are four main things one is your requests
8:17
four main things one is your requests
8:17
four main things one is your requests the total number of requests that is
8:19
the total number of requests that is
8:19
the total number of requests that is taking edits all your availability
8:21
taking edits all your availability
8:21
taking edits all your availability metrics duration metrics which are your
8:24
metrics duration metrics which are your
8:24
metrics duration metrics which are your request latency metrics how much time it
8:26
request latency metrics how much time it
8:26
request latency metrics how much time it is taking to serve a request and yes
8:29
is taking to serve a request and yes
8:29
is taking to serve a request and yes saturation metrics which are your CPU
8:32
saturation metrics which are your CPU
8:32
saturation metrics which are your CPU utilization your memory utilization and
8:36
utilization your memory utilization and
8:37
utilization your memory utilization and dis utilization
8:38
dis utilization
8:38
dis utilization metrics next
8:41
metrics next
8:41
metrics next Logs with logs there are mainly two
8:44
Logs with logs there are mainly two
8:44
Logs with logs there are mainly two types audit logs and your debug Logs
8:48
types audit logs and your debug Logs
8:48
types audit logs and your debug Logs with audit logs they are for your mostly
8:51
with audit logs they are for your mostly
8:51
with audit logs they are for your mostly control plane and data plane operations
8:53
control plane and data plane operations
8:53
control plane and data plane operations for example let's say somebody creates a
8:55
for example let's say somebody creates a
8:55
for example let's say somebody creates a VM deletes a VM or updates a VM
8:59
VM deletes a VM or updates a VM
8:59
VM deletes a VM or updates a VM I as a company would want to know what
9:01
I as a company would want to know what
9:01
I as a company would want to know what is happening for example let's say
9:04
is happening for example let's say
9:04
is happening for example let's say there's a threat actor that has entered
9:06
there's a threat actor that has entered
9:06
there's a threat actor that has entered my system and is randomly creating grow
9:09
my system and is randomly creating grow
9:09
my system and is randomly creating grow resources increasing my cloud cost so I
9:13
resources increasing my cloud cost so I
9:13
resources increasing my cloud cost so I would want to definitely monitor these
9:16
would want to definitely monitor these
9:16
would want to definitely monitor these and that is where audit logs do
9:17
and that is where audit logs do
9:17
and that is where audit logs do definitely help us then we have the
9:20
definitely help us then we have the
9:20
definitely help us then we have the debug logs debug logs are things like
9:24
debug logs debug logs are things like
9:24
debug logs debug logs are things like your statices for example a request
9:26
your statices for example a request
9:26
your statices for example a request failed how do you handle that uh
9:30
failed how do you handle that uh
9:30
failed how do you handle that uh what is the reason the request fail this
9:32
what is the reason the request fail this
9:32
what is the reason the request fail this is where stack T is coming to picture
9:34
is where stack T is coming to picture
9:34
is where stack T is coming to picture and also you can add additional metadata
9:37
and also you can add additional metadata
9:37
and also you can add additional metadata that help you further debug
9:39
that help you further debug
9:39
that help you further debug request traces on the other
9:42
request traces on the other
9:42
request traces on the other hand in a cloud distributed system world
9:46
hand in a cloud distributed system world
9:46
hand in a cloud distributed system world where there are ton of
9:48
where there are ton of
9:48
where there are ton of microservices you would want to
9:50
microservices you would want to
9:51
microservices you would want to understand the LIF lifetime of your
9:52
understand the LIF lifetime of your
9:52
understand the LIF lifetime of your request for example let's say a customer
9:55
request for example let's say a customer
9:55
request for example let's say a customer tells you that hey your request duration
9:58
tells you that hey your request duration
9:58
tells you that hey your request duration is high soorry your request latency is
10:00
is high soorry your request latency is
10:00
is high soorry your request latency is high
10:02
high
10:02
high and your request travels to six or seven
10:05
and your request travels to six or seven
10:05
and your request travels to six or seven different
10:06
different
10:06
different microservices if I want to debug where
10:08
microservices if I want to debug where
10:08
microservices if I want to debug where it is taking a lot of time I would want
10:11
it is taking a lot of time I would want
10:11
it is taking a lot of time I would want to
10:13
to
10:13
to understand where all the request travel
10:15
understand where all the request travel
10:15
understand where all the request travel to and what is the latency at each
10:18
to and what is the latency at each
10:18
to and what is the latency at each microservice traces help you with that
10:20
microservice traces help you with that
10:20
microservice traces help you with that it help you identify bottlenecks and and
10:23
it help you identify bottlenecks and and
10:23
it help you identify bottlenecks and and dependencies so that you get a holistic
10:26
dependencies so that you get a holistic
10:26
dependencies so that you get a holistic view of the system so metrix logs and
10:32
view of the system so metrix logs and
10:32
view of the system so metrix logs and traces as a whole give you the entire
10:35
traces as a whole give you the entire
10:35
traces as a whole give you the entire observability story for your
10:38
service now that we have fundamentals in
10:42
service now that we have fundamentals in
10:42
service now that we have fundamentals in place let's look at why the approach
10:45
place let's look at why the approach
10:45
place let's look at why the approach observability needs to change in a
10:48
observability needs to change in a
10:48
observability needs to change in a multicloud
10:52
environment before that some stats on an
10:55
environment before that some stats on an
10:55
environment before that some stats on an average a company using cloud
10:59
average a company using cloud
10:59
average a company using cloud Cloud to offer the services uses at
11:02
Cloud to offer the services uses at
11:02
Cloud to offer the services uses at least 15 different service 15 different
11:05
least 15 different service 15 different
11:05
least 15 different service 15 different Services what I mean by that for example
11:08
Services what I mean by that for example
11:08
Services what I mean by that for example let's say I am a provider that offers
11:12
let's say I am a provider that offers
11:12
let's say I am a provider that offers logging product to customers right where
11:15
logging product to customers right where
11:15
logging product to customers right where customers can come and inest ingest logs
11:18
customers can come and inest ingest logs
11:18
customers can come and inest ingest logs and search to
11:19
and search to
11:19
and search to them what are the services I'll be
11:22
them what are the services I'll be
11:22
them what are the services I'll be leveraging if I'm using Cloud I might
11:24
leveraging if I'm using Cloud I might
11:24
leveraging if I'm using Cloud I might use an API G API Gateway to route my
11:27
use an API G API Gateway to route my
11:27
use an API G API Gateway to route my service to route my request to different
11:29
service to route my request to different
11:29
service to route my request to different Serv Services then I have my load
11:31
Serv Services then I have my load
11:31
Serv Services then I have my load balancer which we use to Route
11:35
balancer which we use to Route
11:35
balancer which we use to Route distribute requests across VMS you have
11:37
distribute requests across VMS you have
11:37
distribute requests across VMS you have the VMS itself then you might use an Aro
11:40
the VMS itself then you might use an Aro
11:40
the VMS itself then you might use an Aro storage queue where you store all the
11:44
storage queue where you store all the
11:44
storage queue where you store all the logs and then you have
11:47
logs and then you have
11:47
logs and then you have additional no SQL databases to store
11:50
additional no SQL databases to store
11:50
additional no SQL databases to store your data then storage
11:55
Network so if we just think about this
11:59
Network so if we just think about this
11:59
Network so if we just think about this we can we have just listed down at least
12:03
we can we have just listed down at least
12:03
we can we have just listed down at least 13 or 14 different Services next a
12:06
13 or 14 different Services next a
12:06
13 or 14 different Services next a company uses at least nine different
12:10
company uses at least nine different
12:10
company uses at least nine different observability or monitoring tools to
12:12
observability or monitoring tools to
12:13
observability or monitoring tools to manage their application infrastructure
12:15
manage their application infrastructure
12:15
manage their application infrastructure and user
12:17
and user
12:17
and user experience by looking at the stats
12:19
experience by looking at the stats
12:19
experience by looking at the stats itself we can quickly understand how
12:22
itself we can quickly understand how
12:22
itself we can quickly understand how complex it becomes as the observe as the
12:26
complex it becomes as the observe as the
12:26
complex it becomes as the observe as the number of moving Parts grow
12:32
so this is the example of a code snippet
12:34
so this is the example of a code snippet
12:34
so this is the example of a code snippet I have where I'm just using the single
12:37
I have where I'm just using the single
12:37
I have where I'm just using the single Cloud so I am using the Azure monitor
12:40
Cloud so I am using the Azure monitor
12:40
Cloud so I am using the Azure monitor SDK to emit metrix to the Azure Cloud
12:46
SDK to emit metrix to the Azure Cloud
12:46
SDK to emit metrix to the Azure Cloud dashboard right so in this case I'm
12:48
dashboard right so in this case I'm
12:48
dashboard right so in this case I'm using a metrix client emitting a data
12:50
using a metrix client emitting a data
12:50
using a metrix client emitting a data point and I can see that on the cloud
12:53
point and I can see that on the cloud
12:53
point and I can see that on the cloud provider dashboard in this case I'm just
12:55
provider dashboard in this case I'm just
12:55
provider dashboard in this case I'm just using one cloud provider everything is
12:58
using one cloud provider everything is
12:58
using one cloud provider everything is pretty smooth I
12:59
pretty smooth I
12:59
pretty smooth I can set up monitors using one of the a
13:03
can set up monitors using one of the a
13:03
can set up monitors using one of the a services and my servic is
13:07
services and my servic is
13:07
services and my servic is right now let's take a look at how this
13:11
right now let's take a look at how this
13:11
right now let's take a look at how this changes in the multicloud
13:15
world so this is the code
13:18
world so this is the code
13:18
world so this is the code where I am using four different Cloud
13:21
where I am using four different Cloud
13:21
where I am using four different Cloud providers right of course this might not
13:23
providers right of course this might not
13:23
providers right of course this might not be practical but let's consider two of
13:26
be practical but let's consider two of
13:27
be practical but let's consider two of them right so so let's say I'm using
13:30
them right so so let's say I'm using
13:30
them right so so let's say I'm using Azure Monitor and Google Cloud
13:33
Azure Monitor and Google Cloud
13:33
Azure Monitor and Google Cloud to emit my metrics and logs so here in
13:38
to emit my metrics and logs so here in
13:38
to emit my metrics and logs so here in my code I have two
13:39
my code I have two
13:39
my code I have two SDK and then on the right what I see is
13:44
SDK and then on the right what I see is
13:44
SDK and then on the right what I see is two different dashboards to
13:47
two different dashboards to
13:47
two different dashboards to emit two different dashboards have to
13:49
emit two different dashboards have to
13:49
emit two different dashboards have to keep track off for emitting metrix and
13:52
keep track off for emitting metrix and
13:52
keep track off for emitting metrix and logs right from just the looks of it we
13:56
logs right from just the looks of it we
13:56
logs right from just the looks of it we can quickly understand
14:00
can quickly understand
14:00
can quickly understand the complexities in cloud and
14:07
operations yeah the complexities in
14:09
operations yeah the complexities in
14:09
operations yeah the complexities in cloud and operations the companies would
14:13
cloud and operations the companies would
14:13
cloud and operations the companies would have so let's take a look at the
14:17
have so let's take a look at the
14:17
have so let's take a look at the complexities right first because I'm
14:20
complexities right first because I'm
14:20
complexities right first because I'm having multiple sdks I have to maintain
14:23
having multiple sdks I have to maintain
14:23
having multiple sdks I have to maintain them either version updates or something
14:27
them either version updates or something
14:27
them either version updates or something regarding the SDK changes have to m M
14:29
regarding the SDK changes have to m M
14:29
regarding the SDK changes have to m M that it is a continuous pain Point
14:34
that it is a continuous pain Point
14:34
that it is a continuous pain Point second I have to understand various
14:37
second I have to understand various
14:37
second I have to understand various Cloud providers
14:39
Cloud providers
14:39
Cloud providers implementations every cloud provider has
14:41
implementations every cloud provider has
14:41
implementations every cloud provider has their own schema has their own way of
14:44
their own schema has their own way of
14:44
their own schema has their own way of representing dashb has their own uh sdks
14:49
representing dashb has their own uh sdks
14:49
representing dashb has their own uh sdks which makes it tricky when I'm on call
14:53
which makes it tricky when I'm on call
14:53
which makes it tricky when I'm on call imagine I being an on call and I have to
14:57
imagine I being an on call and I have to
14:57
imagine I being an on call and I have to go to three or four different run books
14:59
go to three or four different run books
14:59
go to three or four different run books to understand if each Cloud providers
15:02
to understand if each Cloud providers
15:02
to understand if each Cloud providers work and then on parly I'm getting page
15:06
work and then on parly I'm getting page
15:06
work and then on parly I'm getting page for different service for different
15:08
for different service for different
15:09
for different service for different problems right and then the schemas that
15:12
problems right and then the schemas that
15:12
problems right and then the schemas that each provider also offers is different
15:15
each provider also offers is different
15:15
each provider also offers is different for example let's say I have to search
15:17
for example let's say I have to search
15:17
for example let's say I have to search logs and now what I end up is I have to
15:21
logs and now what I end up is I have to
15:21
logs and now what I end up is I have to go to each runbook understand what the
15:24
go to each runbook understand what the
15:24
go to each runbook understand what the schema is in this cloud provider and
15:26
schema is in this cloud provider and
15:26
schema is in this cloud provider and then search that and when
15:29
then search that and when
15:29
then search that and when you and when you're on call and when you
15:33
you and when you're on call and when you
15:33
you and when you're on call and when you are taking a lot of heat and this
15:35
are taking a lot of heat and this
15:35
are taking a lot of heat and this becomes
15:37
becomes
15:37
becomes added this adds more trickiness to your
15:41
added this adds more trickiness to your
15:41
added this adds more trickiness to your dat so how do we solve this how do we
15:46
dat so how do we solve this how do we
15:46
dat so how do we solve this how do we make
15:48
make
15:48
make operations or how do we simplify
15:53
operations if I take a step back what is
15:56
operations if I take a step back what is
15:56
operations if I take a step back what is the problem that I'm seeing here
15:59
the problem that I'm seeing here
15:59
the problem that I'm seeing here because my tools are tied to a specific
16:02
because my tools are tied to a specific
16:02
because my tools are tied to a specific vendor I'm not able to offer a unified
16:05
vendor I'm not able to offer a unified
16:05
vendor I'm not able to offer a unified experience to my employees or to my devs
16:08
experience to my employees or to my devs
16:09
experience to my employees or to my devs right so if I
16:11
right so if I
16:11
right so if I was generic enough and if I design my
16:16
was generic enough and if I design my
16:16
was generic enough and if I design my architecture in a way that I'm agnostic
16:18
architecture in a way that I'm agnostic
16:18
architecture in a way that I'm agnostic to a cloud
16:20
to a cloud
16:20
to a cloud vendor I I can make operation
16:24
vendor I I can make operation
16:24
vendor I I can make operation smooth right so this is where
16:30
Cloud native comes into picture so with
16:33
Cloud native comes into picture so with
16:33
Cloud native comes into picture so with the adoption of cloud growing at a rapid
16:37
the adoption of cloud growing at a rapid
16:37
the adoption of cloud growing at a rapid Pace lot of Open Source tool came into
16:39
Pace lot of Open Source tool came into
16:39
Pace lot of Open Source tool came into picture for example you have your
16:41
picture for example you have your
16:41
picture for example you have your kubernetes
16:43
kubernetes
16:43
kubernetes infrastructure which started 10 years
16:45
infrastructure which started 10 years
16:45
infrastructure which started 10 years ago and now kubernetes has become a deao
16:49
ago and now kubernetes has become a deao
16:49
ago and now kubernetes has become a deao of for running Services similarly there
16:51
of for running Services similarly there
16:51
of for running Services similarly there are bunch of other open source tools as
16:54
are bunch of other open source tools as
16:54
are bunch of other open source tools as well so today what we will talk about is
16:57
well so today what we will talk about is
16:57
well so today what we will talk about is one such open Source tool called open
17:01
one such open Source tool called open
17:01
one such open Source tool called open Telemetry right so what is open
17:05
Telemetry right so what is open
17:05
Telemetry right so what is open Telemetry simply put it is an open-
17:08
Telemetry simply put it is an open-
17:08
Telemetry simply put it is an open- Source standard and
17:10
Source standard and
17:10
Source standard and specification to emit your metrix logs
17:14
specification to emit your metrix logs
17:14
specification to emit your metrix logs and
17:16
and
17:16
and traces right so what are and also the
17:21
traces right so what are and also the
17:21
traces right so what are and also the major Cloud providers like
17:24
major Cloud providers like
17:24
major Cloud providers like AWS Azure
17:27
AWS Azure
17:27
AWS Azure gcp and orle already support this
17:30
gcp and orle already support this
17:30
gcp and orle already support this project this is a not Cloud native
17:32
project this is a not Cloud native
17:32
project this is a not Cloud native Computing Foundation project and many of
17:35
Computing Foundation project and many of
17:35
Computing Foundation project and many of the cloud providers do already support
17:36
the cloud providers do already support
17:36
the cloud providers do already support this and the best part it is vendor
17:41
this and the best part it is vendor
17:41
this and the best part it is vendor agnostic you are not tied up to any
17:44
agnostic you are not tied up to any
17:45
agnostic you are not tied up to any specific Cloud vendor which is very
17:47
specific Cloud vendor which is very
17:47
specific Cloud vendor which is very critical when you think about a cloud
17:50
critical when you think about a cloud
17:50
critical when you think about a cloud which you think when you think about a
17:52
which you think when you think about a
17:52
which you think when you think about a multi Cloud
17:55
multi Cloud
17:55
multi Cloud architecture so let's look
17:57
architecture so let's look
17:57
architecture so let's look at let's get get deep into the open
18:00
at let's get get deep into the open
18:00
at let's get get deep into the open Telemetry
18:01
Telemetry
18:01
Telemetry world what does open Telemetry have so
18:04
world what does open Telemetry have so
18:04
world what does open Telemetry have so first it has a specification
18:06
first it has a specification
18:06
first it has a specification specification talks about what I need to
18:10
specification talks about what I need to
18:10
specification talks about what I need to do when I want to implement open tary in
18:13
do when I want to implement open tary in
18:13
do when I want to implement open tary in a specific language this we don't we as
18:17
a specific language this we don't we as
18:17
a specific language this we don't we as developers don't deal with this
18:19
developers don't deal with this
18:19
developers don't deal with this regularly this comes into picture when
18:23
regularly this comes into picture when
18:23
regularly this comes into picture when you want to implement open in a specific
18:25
you want to implement open in a specific
18:25
you want to implement open in a specific language right and by the way open
18:29
language right and by the way open
18:29
language right and by the way open Telemetry already supports bunch of
18:31
Telemetry already supports bunch of
18:31
Telemetry already supports bunch of different languages go C Java Ruby all
18:35
different languages go C Java Ruby all
18:35
different languages go C Java Ruby all these
18:37
these
18:37
these languages next semantics you remember
18:40
languages next semantics you remember
18:40
languages next semantics you remember the problem we discussed about not
18:42
the problem we discussed about not
18:42
the problem we discussed about not having a common schema with
18:47
having a common schema with
18:47
having a common schema with semantics we can define common schemas
18:50
semantics we can define common schemas
18:50
semantics we can define common schemas that get propagated different Cloud
18:52
that get propagated different Cloud
18:52
that get propagated different Cloud providers which will help you querying
18:55
providers which will help you querying
18:55
providers which will help you querying your metric logs or traces e
18:59
your metric logs or traces e
18:59
your metric logs or traces e and then we have the SDK
19:02
and then we have the SDK
19:02
and then we have the SDK itself this is what we will use to emit
19:06
itself this is what we will use to emit
19:06
itself this is what we will use to emit the metrix
19:08
the metrix
19:08
the metrix logs to yeah to emit your observability
19:12
logs to yeah to emit your observability
19:12
logs to yeah to emit your observability Matrix and
19:13
Matrix and
19:13
Matrix and logs Fin and next there is an exporter
19:17
logs Fin and next there is an exporter
19:17
logs Fin and next there is an exporter this is a fundamental piece in this
19:20
this is a fundamental piece in this
19:20
this is a fundamental piece in this whole
19:21
whole
19:21
whole architecture once we
19:23
architecture once we
19:24
architecture once we emit the S once we emit metrix logs and
19:27
emit the S once we emit metrix logs and
19:27
emit the S once we emit metrix logs and traces using the SDK
19:29
traces using the SDK
19:29
traces using the SDK each cloud provider has their own
19:32
each cloud provider has their own
19:32
each cloud provider has their own exporter Azure monitor has their own
19:34
exporter Azure monitor has their own
19:34
exporter Azure monitor has their own exporter AWS Google cloud data dog sekin
19:39
exporter AWS Google cloud data dog sekin
19:39
exporter AWS Google cloud data dog sekin all these have on
19:40
all these have on
19:40
all these have on exporters these exporters query that
19:43
exporters these exporters query that
19:43
exporters these exporters query that open perimetry SDK and then export those
19:48
open perimetry SDK and then export those
19:48
open perimetry SDK and then export those things into their own backends and that
19:50
things into their own backends and that
19:50
things into their own backends and that is where we talk about the final
19:54
is where we talk about the final
19:54
is where we talk about the final piece where back end
19:56
piece where back end
19:56
piece where back end is the Google every cloud provider's own
20:00
is the Google every cloud provider's own
20:00
is the Google every cloud provider's own implementation for example Cloud watch
20:02
implementation for example Cloud watch
20:02
implementation for example Cloud watch can be AWS black and Azure monitor can
20:04
can be AWS black and Azure monitor can
20:04
can be AWS black and Azure monitor can be azure's backend and Google cloud has
20:06
be azure's backend and Google cloud has
20:06
be azure's backend and Google cloud has similarly its own
20:09
similarly its own
20:09
similarly its own back to summarize there are five
20:12
back to summarize there are five
20:12
back to summarize there are five different components you know in the
20:14
different components you know in the
20:15
different components you know in the architecture here one is your
20:17
architecture here one is your
20:17
architecture here one is your specification mostly comes into picture
20:20
specification mostly comes into picture
20:20
specification mostly comes into picture when you implement a specific language
20:22
when you implement a specific language
20:22
when you implement a specific language semantics that help you define a common
20:24
semantics that help you define a common
20:24
semantics that help you define a common schema the SDK itself which you use to
20:28
schema the SDK itself which you use to
20:28
schema the SDK itself which you use to emit metrics logs and Tres exporter
20:32
emit metrics logs and Tres exporter
20:32
emit metrics logs and Tres exporter which the cloud providers already
20:35
which the cloud providers already
20:35
which the cloud providers already provide and they are open
20:36
provide and they are open
20:37
provide and they are open sourc you use uh and you use the
20:40
sourc you use uh and you use the
20:40
sourc you use uh and you use the exporters to send the metrix logs and
20:42
exporters to send the metrix logs and
20:42
exporters to send the metrix logs and Tres to the backends and the back ends
20:45
Tres to the backends and the back ends
20:45
Tres to the backends and the back ends itself now let's take a look at the
20:50
itself now let's take a look at the
20:50
itself now let's take a look at the architecture we'll take a look at
20:54
architecture we'll take a look at
20:54
architecture we'll take a look at how in in the word before open Telemetry
20:58
how in in the word before open Telemetry
20:58
how in in the word before open Telemetry the architect is and
21:01
the architect is and
21:01
the architect is and currently with open Telemetry how you
21:03
currently with open Telemetry how you
21:03
currently with open Telemetry how you can change
21:07
this so on the left you see that you
21:11
this so on the left you see that you
21:11
this so on the left you see that you have a
21:12
have a
21:12
have a VM the VM has your service and the
21:16
VM the VM has your service and the
21:16
VM the VM has your service and the service is using the Azure monitor SDK
21:19
service is using the Azure monitor SDK
21:19
service is using the Azure monitor SDK to emit metrics to Azure right now from
21:22
to emit metrics to Azure right now from
21:22
to emit metrics to Azure right now from whatever fundamentals and whatever
21:24
whatever fundamentals and whatever
21:24
whatever fundamentals and whatever discussion we had till now what we
21:26
discussion we had till now what we
21:27
discussion we had till now what we understand is
21:29
understand is
21:29
understand is tying up the service to the SDK in a
21:33
tying up the service to the SDK in a
21:33
tying up the service to the SDK in a multicloud world is causes the
21:37
multicloud world is causes the
21:37
multicloud world is causes the trickiness here what we are doing is
21:39
trickiness here what we are doing is
21:39
trickiness here what we are doing is replacing the SDK with the open
21:41
replacing the SDK with the open
21:42
replacing the SDK with the open Telemetry SDK right the VM is same the
21:46
Telemetry SDK right the VM is same the
21:46
Telemetry SDK right the VM is same the service is
21:47
service is
21:47
service is same but the SDK itself changes and then
21:52
same but the SDK itself changes and then
21:52
same but the SDK itself changes and then we
21:53
we
21:53
we have the cloud specific
21:56
have the cloud specific
21:56
have the cloud specific exporter to clarify this is not any side
22:00
exporter to clarify this is not any side
22:00
exporter to clarify this is not any side card or the exporter is not a side card
22:03
card or the exporter is not a side card
22:03
card or the exporter is not a side card it is just another package in your cop
22:06
it is just another package in your cop
22:06
it is just another package in your cop or a maven dependency in your pops
22:11
or a maven dependency in your pops
22:11
or a maven dependency in your pops right as part of your service you emit
22:14
right as part of your service you emit
22:14
right as part of your service you emit metric you emit the observability pieces
22:18
metric you emit the observability pieces
22:18
metric you emit the observability pieces using the open Telemetry SDK and the
22:22
using the open Telemetry SDK and the
22:22
using the open Telemetry SDK and the exporters extract those
22:25
exporters extract those
22:25
exporters extract those pieces do any necessary processing they
22:28
pieces do any necessary processing they
22:28
pieces do any necessary processing they need on them and then send it to the
22:32
need on them and then send it to the
22:32
need on them and then send it to the corresponding packs in this case
22:36
corresponding packs in this case
22:36
corresponding packs in this case a
22:38
a
22:38
a right
22:41
so this is the Cod snippet
22:45
so this is the Cod snippet
22:45
so this is the Cod snippet from when I'm using open ter imetry so
22:49
from when I'm using open ter imetry so
22:49
from when I'm using open ter imetry so what I have here is a meter and a
22:52
what I have here is a meter and a
22:52
what I have here is a meter and a counter right these are the name spaces
22:55
counter right these are the name spaces
22:55
counter right these are the name spaces that belong to
22:59
open elry and
23:01
open elry and
23:01
open elry and then I am adding
23:05
then I am adding
23:05
then I am adding exports something like this so the
23:07
exports something like this so the
23:07
exports something like this so the exporter we discussed previously we are
23:10
exporter we discussed previously we are
23:10
exporter we discussed previously we are adding it like this like a Prometheus
23:13
adding it like this like a Prometheus
23:13
adding it like this like a Prometheus exporter and a console exporter
23:14
exporter and a console exporter
23:14
exporter and a console exporter similarly I can
23:17
similarly I can
23:17
similarly I can add Azure
23:19
add Azure
23:19
add Azure exporter CL Google Cloud exporter
23:22
exporter CL Google Cloud exporter
23:22
exporter CL Google Cloud exporter right and the amazing part is if I see
23:26
right and the amazing part is if I see
23:26
right and the amazing part is if I see here my the way how I metric does not
23:29
here my the way how I metric does not
23:29
here my the way how I metric does not change so if I compare my slide to the
23:32
change so if I compare my slide to the
23:32
change so if I compare my slide to the previous one where we had four different
23:35
previous one where we had four different
23:35
previous one where we had four different sdks see the simplification that
23:39
sdks see the simplification that
23:39
sdks see the simplification that this infrastructure helps
23:42
this infrastructure helps
23:42
this infrastructure helps us and tomorrow if I want to extend it
23:45
us and tomorrow if I want to extend it
23:45
us and tomorrow if I want to extend it to a different cloud provider I just
23:47
to a different cloud provider I just
23:47
to a different cloud provider I just need to add the exporter and an end
23:49
need to add the exporter and an end
23:49
need to add the exporter and an end point right that's it I have to do I
23:52
point right that's it I have to do I
23:52
point right that's it I have to do I don't have to change any
23:55
don't have to change any
23:55
don't have to change any schema any name space or anything of
23:59
schema any name space or anything of
23:59
schema any name space or anything of that sort and this also helps you
24:02
that sort and this also helps you
24:02
that sort and this also helps you abstract these pieces of into a library
24:05
abstract these pieces of into a library
24:05
abstract these pieces of into a library so that you don't have to touch these
24:07
so that you don't have to touch these
24:07
so that you don't have to touch these pieces every time improving your
24:11
pieces every time improving your
24:11
pieces every time improving your maintainability in the
24:13
maintainability in the
24:13
maintainability in the code so this this is our SDK concerns
24:17
code so this this is our SDK concerns
24:18
code so this this is our SDK concerns right
24:21
now this is
24:23
now this is
24:23
now this is how the scenario would play out in when
24:27
how the scenario would play out in when
24:27
how the scenario would play out in when you're using mod than one Cloud right in
24:31
you're using mod than one Cloud right in
24:32
you're using mod than one Cloud right in all the three cases here what I have is
24:36
all the three cases here what I have is
24:36
all the three cases here what I have is the VM service and the SDK none of these
24:40
the VM service and the SDK none of these
24:40
the VM service and the SDK none of these change the only things that change is a
24:43
change the only things that change is a
24:43
change the only things that change is a package which is your
24:47
package which is your
24:47
package which is your export right these are the only things
24:49
export right these are the only things
24:49
export right these are the only things that change and then of course the back
24:51
that change and then of course the back
24:51
that change and then of course the back ends if wherever you want to
24:54
ends if wherever you want to
24:55
ends if wherever you want to export another thing that we discussed
24:58
export another thing that we discussed
24:58
export another thing that we discussed is is not having a seamless experience
25:00
is is not having a seamless experience
25:00
is is not having a seamless experience for
25:03
for
25:03
for your dashboard dashboards right this is
25:05
your dashboard dashboards right this is
25:05
your dashboard dashboards right this is where grafana comes into
25:07
where grafana comes into
25:07
where grafana comes into picture what you can basically do
25:11
picture what you can basically do
25:11
picture what you can basically do is with
25:13
is with
25:13
is with grafana configure data sources that
25:16
grafana configure data sources that
25:16
grafana configure data sources that belong to each Cloud for example AWS has
25:20
belong to each Cloud for example AWS has
25:20
belong to each Cloud for example AWS has its own data source Azure has its own
25:22
its own data source Azure has its own
25:22
its own data source Azure has its own data source GTP has its own data
25:24
data source GTP has its own data
25:24
data source GTP has its own data source and what grafana will do is it
25:27
source and what grafana will do is it
25:27
source and what grafana will do is it export all those things into its own
25:30
export all those things into its own
25:30
export all those things into its own dashboard and provide one seamless
25:34
dashboard and provide one seamless
25:34
dashboard and provide one seamless experience so where did we start we
25:37
experience so where did we start we
25:37
experience so where did we start we started with not having a unified
25:39
started with not having a unified
25:39
started with not having a unified experience having
25:42
experience having
25:42
experience having multiple sdks and multiple CL multiple
25:45
multiple sdks and multiple CL multiple
25:45
multiple sdks and multiple CL multiple dashboards right to now where we are is
25:49
dashboards right to now where we are is
25:49
dashboards right to now where we are is one seamless experience just different
25:52
one seamless experience just different
25:52
one seamless experience just different Prov exporters which are already
25:54
Prov exporters which are already
25:54
Prov exporters which are already provided to us by the cloud providers
25:57
provided to us by the cloud providers
25:57
provided to us by the cloud providers and then one single dashboard see how we
26:00
and then one single dashboard see how we
26:00
and then one single dashboard see how we have streamlined the pipeline to make
26:03
have streamlined the pipeline to make
26:03
have streamlined the pipeline to make the experience much better for our devs
26:06
the experience much better for our devs
26:06
the experience much better for our devs now they are very much Happy
26:11
right so this is how you can provide one
26:15
right so this is how you can provide one
26:15
right so this is how you can provide one seamless experience and decrease the
26:19
seamless experience and decrease the
26:19
seamless experience and decrease the pain that operations have to go
26:24
through next so this is a good to know
26:29
through next so this is a good to know
26:29
through next so this is a good to know not you don't have to we as developers
26:31
not you don't have to we as developers
26:31
not you don't have to we as developers don't deal with this regularly this is
26:33
don't deal with this regularly this is
26:33
don't deal with this regularly this is how Cloud providers Implement open Tate
26:36
how Cloud providers Implement open Tate
26:36
how Cloud providers Implement open Tate right so on the left what you see
26:39
right so on the left what you see
26:39
right so on the left what you see is every Everything of this sort Remains
26:42
is every Everything of this sort Remains
26:42
is every Everything of this sort Remains the Same but instead of talking to the
26:45
the Same but instead of talking to the
26:45
the Same but instead of talking to the Azure back end itself the cloud provider
26:47
Azure back end itself the cloud provider
26:47
Azure back end itself the cloud provider talks to an agent and the agent is the
26:50
talks to an agent and the agent is the
26:50
talks to an agent and the agent is the one that exports exports metric to the
26:54
one that exports exports metric to the
26:54
one that exports exports metric to the Azure packet right so this is
26:58
Azure packet right so this is
26:58
Azure packet right so this is normally what ends up happening is if I
27:01
normally what ends up happening is if I
27:01
normally what ends up happening is if I want to send any of the observability
27:03
want to send any of the observability
27:03
want to send any of the observability stats from my VM to a
27:08
stats from my VM to a
27:08
stats from my VM to a machine agents are the one that used to
27:10
machine agents are the one that used to
27:10
machine agents are the one that used to do the job right now I also want to
27:13
do the job right now I also want to
27:13
do the job right now I also want to provide backo support supportability and
27:16
provide backo support supportability and
27:16
provide backo support supportability and also support the new ways of doing this
27:19
also support the new ways of doing this
27:19
also support the new ways of doing this things so this is a good way to this a
27:22
things so this is a good way to this a
27:22
things so this is a good way to this a good segue where I'm also supporting the
27:24
good segue where I'm also supporting the
27:24
good segue where I'm also supporting the current infrastructure and the open
27:26
current infrastructure and the open
27:26
current infrastructure and the open Telemetry infrastructure as a
27:31
on the second h on the second
27:32
on the second h on the second
27:32
on the second h on the second architecture diagram what you see
27:36
architecture diagram what you see
27:36
architecture diagram what you see is an collector right so collector is
27:40
is an collector right so collector is
27:40
is an collector right so collector is nothing but a combination of multiple
27:43
nothing but a combination of multiple
27:43
nothing but a combination of multiple processors and multiple exporters
27:47
processors and multiple exporters
27:47
processors and multiple exporters right what I can do
27:50
right what I can do
27:50
right what I can do is so this because the exporter because
27:55
is so this because the exporter because
27:55
is so this because the exporter because this collector is an executable
27:59
this collector is an executable
27:59
this collector is an executable it might end up taking lot of space on
28:02
it might end up taking lot of space on
28:02
it might end up taking lot of space on your VM or lot of memory on your VM
28:04
your VM or lot of memory on your VM
28:04
your VM or lot of memory on your VM let's say I want to limit that that is
28:06
let's say I want to limit that that is
28:06
let's say I want to limit that that is when I can have a processor to limit the
28:09
when I can have a processor to limit the
28:09
when I can have a processor to limit the usage of memory for this collector
28:13
usage of memory for this collector
28:13
usage of memory for this collector similarly let's
28:15
similarly let's
28:15
similarly let's say when when I'm a cloud provider with
28:19
say when when I'm a cloud provider with
28:19
say when when I'm a cloud provider with the Telemetry the customers are
28:21
the Telemetry the customers are
28:21
the Telemetry the customers are providing me also want to add some
28:23
providing me also want to add some
28:23
providing me also want to add some additional metad data I want to do some
28:25
additional metad data I want to do some
28:25
additional metad data I want to do some massaging on the data that is when I
28:28
massaging on the data that is when I
28:28
massaging on the data that is when I would also add additional processors so
28:32
would also add additional processors so
28:32
would also add additional processors so if I think about it if I think about it
28:34
if I think about it if I think about it
28:34
if I think about it if I think about it like a big data pipeline I have the
28:36
like a big data pipeline I have the
28:36
like a big data pipeline I have the source I have multiple processors and
28:40
source I have multiple processors and
28:40
source I have multiple processors and then I have multiple exporters and each
28:43
then I have multiple exporters and each
28:43
then I have multiple exporters and each exporter can send data to one of the
28:46
exporter can send data to one of the
28:46
exporter can send data to one of the backends right so this is one more
28:50
backends right so this is one more
28:50
backends right so this is one more architecture
28:58
to
29:01
conclude what I would say is if if
29:04
conclude what I would say is if if
29:04
conclude what I would say is if if companies want to adopt multic Clause
29:06
companies want to adopt multic Clause
29:06
companies want to adopt multic Clause strategy and if they want to grow in
29:09
strategy and if they want to grow in
29:09
strategy and if they want to grow in that space being vendor agnostic will
29:12
that space being vendor agnostic will
29:12
that space being vendor agnostic will help them definitely skip if it is
29:15
help them definitely skip if it is
29:15
help them definitely skip if it is single Cloud then it does not matter
29:17
single Cloud then it does not matter
29:17
single Cloud then it does not matter much but vendor being vender agnostic
29:20
much but vendor being vender agnostic
29:20
much but vendor being vender agnostic will help them scale and and help them
29:24
will help them scale and and help them
29:25
will help them scale and and help them offer their customers better experience
29:28
offer their customers better experience
29:28
offer their customers better experience I and ship ship features
29:32
I and ship ship features
29:32
I and ship ship features faster then you have your apart from
29:36
faster then you have your apart from
29:36
faster then you have your apart from these open source toolings there are
29:37
these open source toolings there are
29:37
these open source toolings there are some other open source toolings as well
29:39
some other open source toolings as well
29:39
some other open source toolings as well for example with Prometheus you can set
29:42
for example with Prometheus you can set
29:42
for example with Prometheus you can set up or emit metrics Loki is also similar
29:46
up or emit metrics Loki is also similar
29:46
up or emit metrics Loki is also similar to promus but for logs this is a plug-in
29:48
to promus but for logs this is a plug-in
29:49
to promus but for logs this is a plug-in by grafana and then you have zapin which
29:53
by grafana and then you have zapin which
29:53
by grafana and then you have zapin which is responsible for your traces so all
29:57
is responsible for your traces so all
29:57
is responsible for your traces so all these have corresponding exporters and
29:58
these have corresponding exporters and
29:58
these have corresponding exporters and you can just send them send the data to
30:02
you can just send them send the data to
30:02
you can just send them send the data to the backends
30:04
the backends
30:04
the backends right with that I would like to conclude
30:08
right with that I would like to conclude
30:08
right with that I would like to conclude my talk thank you everyone for taking
30:10
my talk thank you everyone for taking
30:10
my talk thank you everyone for taking the time to listen to me if you if you
30:13
the time to listen to me if you if you
30:13
the time to listen to me if you if you have any additional feedback please feel
30:16
have any additional feedback please feel
30:16
have any additional feedback please feel free to reach out me thank you
30:19
free to reach out me thank you
30:19
free to reach out me thank you [Music]
30:28
he