Up next in 10
Crafting Resilient Integration Solutions | Software Architecture Conference
0 views
Aug 6, 2025
"In today's interconnected world, integrating diverse systems, data sources, APIs and legacy apps is very challenging for companies. In this talk will explore multiple integration patterns such as data aggregation, real-time synchronization, event broadcasting, data orchestration and more. We'll explore how these patterns address various architectural challenges and drive operational efficiency. This session will provide a technical deep dive into each pattern, discussing their implementation, benefits, and potential challenges. Attendees will gain a comprehensive understanding of how to architect robust, resilient and scalable integration solutions. Finally, we'll showcase MuleSoft Anypoint Platform as an exemple of a solution that supports these patterns. Showing how its comprehensive tools and components can simplify integration and enhance connectivity." 🔗 Conference Website: https://softwarearchitecture.live 📺 CSharp TV - Dev Streaming Destination http://csharp.tv 🌎 C# Corner - Community of Software and Data Developers https://www.c-sharpcorner.com #CSharpTV #CSharpCorner #CSharp #SoftwareArchitectureConf
View Video Transcript
0:03
so my name is Awan I'm CIO Academy at at
0:07
so my name is Awan I'm CIO Academy at at
0:07
so my name is Awan I'm CIO Academy at at Serv group and today we will talk about
0:09
Serv group and today we will talk about
0:09
Serv group and today we will talk about how to craft resilience integration
0:11
how to craft resilience integration
0:11
how to craft resilience integration solution and uh really why I chose to
0:13
solution and uh really why I chose to
0:13
solution and uh really why I chose to talk about this subject is because it's
0:15
talk about this subject is because it's
0:15
talk about this subject is because it's it's my it becomes my daily work and uh
0:19
it's my it becomes my daily work and uh
0:19
it's my it becomes my daily work and uh I and I I always wanted to share my to
0:22
I and I I always wanted to share my to
0:22
I and I I always wanted to share my to share my experience and my know my
0:24
share my experience and my know my
0:24
share my experience and my know my olders with with other with others in
0:26
olders with with other with others in
0:26
olders with with other with others in the community and it will be my first
0:28
the community and it will be my first
0:28
the community and it will be my first time that I will give this talk so I'm
0:30
time that I will give this talk so I'm
0:30
time that I will give this talk so I'm so sorry if there is something missing
0:32
so sorry if there is something missing
0:32
so sorry if there is something missing or something like that and you so we
0:34
or something like that and you so we
0:34
or something like that and you so we will start now so so before starting our
0:37
will start now so so before starting our
0:37
will start now so so before starting our talk uh about integration Solutions we
0:39
talk uh about integration Solutions we
0:39
talk uh about integration Solutions we should establish some context so the
0:41
should establish some context so the
0:41
should establish some context so the first question to ask is what brought us
0:43
first question to ask is what brought us
0:43
first question to ask is what brought us to here why we are talking about this
0:45
to here why we are talking about this
0:45
to here why we are talking about this subject the answer is not simple but
0:49
subject the answer is not simple but
0:49
subject the answer is not simple but because it depends on problems that
0:51
because it depends on problems that
0:51
because it depends on problems that perhaps mod developers have not faced so
0:53
perhaps mod developers have not faced so
0:54
perhaps mod developers have not faced so if you are working in a large company
0:56
if you are working in a large company
0:56
if you are working in a large company you may con constantly wonder how you
0:58
you may con constantly wonder how you
0:59
you may con constantly wonder how you arrived at the Que State and what and
1:02
arrived at the Que State and what and
1:02
arrived at the Que State and what and what may have been overlooked if you're
1:04
what may have been overlooked if you're
1:04
what may have been overlooked if you're an architect or someone with a role
1:07
an architect or someone with a role
1:07
an architect or someone with a role that's complex to Define by examining
1:09
that's complex to Define by examining
1:09
that's complex to Define by examining multiple projects with your within your
1:11
multiple projects with your within your
1:11
multiple projects with your within your company you will notice that
1:13
company you will notice that
1:13
company you will notice that applications have cision among them yet
1:16
applications have cision among them yet
1:16
applications have cision among them yet they are designed as if this isn't the
1:18
they are designed as if this isn't the
1:18
they are designed as if this isn't the case this is often due to lack of
1:21
case this is often due to lack of
1:21
case this is often due to lack of long-term Vision if these applications
1:23
long-term Vision if these applications
1:23
long-term Vision if these applications need to share information with each
1:25
need to share information with each
1:25
need to share information with each other we are faced with a difficult
1:28
other we are faced with a difficult
1:28
other we are faced with a difficult Technical and management process
1:30
Technical and management process
1:30
Technical and management process so because each app belongs to a
1:32
so because each app belongs to a
1:33
so because each app belongs to a different department and may be
1:34
different department and may be
1:34
different department and may be developed using different programming
1:36
developed using different programming
1:36
developed using different programming languages and toos here I'm trying to
1:39
languages and toos here I'm trying to
1:39
languages and toos here I'm trying to highlight the importance of service or
1:41
highlight the importance of service or
1:41
highlight the importance of service or oriented architecture so perhaps members
1:44
oriented architecture so perhaps members
1:44
oriented architecture so perhaps members of the hierarchy are hesitating to
1:46
of the hierarchy are hesitating to
1:46
of the hierarchy are hesitating to collaborate and discuss their apps but
1:49
collaborate and discuss their apps but
1:49
collaborate and discuss their apps but they need to because they should offer a
1:51
they need to because they should offer a
1:51
they need to because they should offer a good service to the user at the minimum
1:54
good service to the user at the minimum
1:54
good service to the user at the minimum cost otherwise another company will do
1:58
cost otherwise another company will do
1:58
cost otherwise another company will do so when we are talking about sharing
2:00
so when we are talking about sharing
2:00
so when we are talking about sharing information between apps between apps
2:03
information between apps between apps
2:03
information between apps between apps we're talking about Services we're not
2:04
we're talking about Services we're not
2:04
we're talking about Services we're not referring to someone who will write an
2:06
referring to someone who will write an
2:06
referring to someone who will write an information on a sheet of paper and send
2:09
information on a sheet of paper and send
2:09
information on a sheet of paper and send it to a member of another team who will
2:11
it to a member of another team who will
2:11
it to a member of another team who will then manually input input it in into
2:14
then manually input input it in into
2:14
then manually input input it in into their app that method it's not uh
2:17
their app that method it's not uh
2:17
their app that method it's not uh inefficient that's it's not efficient
2:19
inefficient that's it's not efficient
2:19
inefficient that's it's not efficient okay that's not what we are considering
2:21
okay that's not what we are considering
2:21
okay that's not what we are considering we are trying to find a modern way to
2:24
we are trying to find a modern way to
2:24
we are trying to find a modern way to transfer messages that need to be secure
2:27
transfer messages that need to be secure
2:27
transfer messages that need to be secure reliable monitorable manageable and and
2:30
reliable monitorable manageable and and
2:30
reliable monitorable manageable and and finally loely coupled it means that we
2:32
finally loely coupled it means that we
2:32
finally loely coupled it means that we should integrate our applications into a
2:35
should integrate our applications into a
2:35
should integrate our applications into a system so that they can communicate and
2:37
system so that they can communicate and
2:37
system so that they can communicate and share messages but do you think that we
2:40
share messages but do you think that we
2:40
share messages but do you think that we started with a perfect solution of
2:41
started with a perfect solution of
2:42
started with a perfect solution of course not so let's take a deep dive
2:44
course not so let's take a deep dive
2:44
course not so let's take a deep dive into history we will start with P2P
2:46
into history we will start with P2P
2:46
into history we will start with P2P integration after that we will talk
2:47
integration after that we will talk
2:48
integration after that we will talk about Central e broker and we'll finish
2:50
about Central e broker and we'll finish
2:50
about Central e broker and we'll finish with Enterprise service bus so point to
2:53
with Enterprise service bus so point to
2:53
with Enterprise service bus so point to point integration was the first method
2:55
point integration was the first method
2:55
point integration was the first method employed to achieve integration between
2:57
employed to achieve integration between
2:57
employed to achieve integration between apps before explaining how it works here
3:00
apps before explaining how it works here
3:00
apps before explaining how it works here is a figure that illustrates how pointto
3:02
is a figure that illustrates how pointto
3:02
is a figure that illustrates how pointto Point architecture functions the point
3:04
Point architecture functions the point
3:04
Point architecture functions the point topoint integration approach connects
3:05
topoint integration approach connects
3:05
topoint integration approach connects two apps with a direct relationship
3:08
two apps with a direct relationship
3:08
two apps with a direct relationship imagine we want to establish a
3:09
imagine we want to establish a
3:09
imagine we want to establish a communication between house two and
3:11
communication between house two and
3:11
communication between house two and house house one and house two using a
3:13
house house one and house two using a
3:13
house house one and house two using a P2P connection we would have a direct
3:15
P2P connection we would have a direct
3:16
P2P connection we would have a direct line between the two it means that if
3:18
line between the two it means that if
3:18
line between the two it means that if the line goes down we will lose the
3:20
the line goes down we will lose the
3:20
the line goes down we will lose the connection between the two houses
3:23
connection between the two houses
3:23
connection between the two houses therefore we cannot confidently say that
3:25
therefore we cannot confidently say that
3:25
therefore we cannot confidently say that we have a reliable system if we wanted
3:27
we have a reliable system if we wanted
3:28
we have a reliable system if we wanted to add a house three we would need to
3:30
to add a house three we would need to
3:30
to add a house three we would need to add additional lines between houses one
3:33
add additional lines between houses one
3:33
add additional lines between houses one and three and two and and three this
3:36
and three and two and and three this
3:36
and three and two and and three this means that our houses are tightly
3:39
means that our houses are tightly
3:39
means that our houses are tightly coupled it's also insecure because we
3:42
coupled it's also insecure because we
3:42
coupled it's also insecure because we have a direct relationship which becomes
3:44
have a direct relationship which becomes
3:44
have a direct relationship which becomes a single point of attack for spying on
3:47
a single point of attack for spying on
3:47
a single point of attack for spying on our data moreover it's not monitorable
3:50
our data moreover it's not monitorable
3:50
our data moreover it's not monitorable okay because we can't easily keep track
3:52
okay because we can't easily keep track
3:52
okay because we can't easily keep track of our communication due to the lack of
3:54
of our communication due to the lack of
3:54
of our communication due to the lack of a central monitoring system and finally
3:58
a central monitoring system and finally
3:58
a central monitoring system and finally it's not managing
4:00
it's not managing
4:00
it's not managing because maning direct lines between a
4:02
because maning direct lines between a
4:02
because maning direct lines between a large number of houses is challenging
4:05
large number of houses is challenging
4:05
large number of houses is challenging now we can draw a parall between houses
4:07
now we can draw a parall between houses
4:07
now we can draw a parall between houses and a service or oriented architecture
4:09
and a service or oriented architecture
4:09
and a service or oriented architecture for instance as seen in this diagram
4:12
for instance as seen in this diagram
4:12
for instance as seen in this diagram service CRM is directly coupled to HR HR
4:16
service CRM is directly coupled to HR HR
4:16
service CRM is directly coupled to HR HR is directly connected to finance this
4:18
is directly connected to finance this
4:18
is directly connected to finance this setup is risky because if we make
4:20
setup is risky because if we make
4:20
setup is risky because if we make changes in the finance service we must
4:22
changes in the finance service we must
4:22
changes in the finance service we must initiate a new integration process to
4:24
initiate a new integration process to
4:24
initiate a new integration process to adapt to all other
4:27
adapt to all other
4:27
adapt to all other services and that's a laborious task to
4:30
services and that's a laborious task to
4:30
services and that's a laborious task to undertake each time we need to make
4:32
undertake each time we need to make
4:32
undertake each time we need to make modifications to the services so is this
4:35
modifications to the services so is this
4:35
modifications to the services so is this approach still applicable clearly not
4:38
approach still applicable clearly not
4:38
approach still applicable clearly not after P2P we moved on to something
4:41
after P2P we moved on to something
4:41
after P2P we moved on to something called Central EI okay Enterprise
4:43
called Central EI okay Enterprise
4:43
called Central EI okay Enterprise application integration broker which is
4:46
application integration broker which is
4:46
application integration broker which is better approach let's explore this in
4:48
better approach let's explore this in
4:48
better approach let's explore this in the following slide so to be clear this
4:51
the following slide so to be clear this
4:51
the following slide so to be clear this approach resoled all the pointto point
4:55
approach resoled all the pointto point
4:55
approach resoled all the pointto point issues by connecting applications
4:57
issues by connecting applications
4:57
issues by connecting applications through a centralized system and that
4:59
through a centralized system and that
4:59
through a centralized system and that the KE word centralized so if we want to
5:02
the KE word centralized so if we want to
5:02
the KE word centralized so if we want to benefit from approach the leaders of
5:05
benefit from approach the leaders of
5:05
benefit from approach the leaders of each project or service must give up
5:07
each project or service must give up
5:07
each project or service must give up control to a centralized infrastructure
5:10
control to a centralized infrastructure
5:10
control to a centralized infrastructure okay where each service is integrated
5:12
okay where each service is integrated
5:12
okay where each service is integrated thus our system will be secure reliable
5:16
thus our system will be secure reliable
5:16
thus our system will be secure reliable Loosely coupled monitorable and
5:17
Loosely coupled monitorable and
5:17
Loosely coupled monitorable and manageable but we will lose some control
5:20
manageable but we will lose some control
5:20
manageable but we will lose some control over our services perhaps this seems a
5:23
over our services perhaps this seems a
5:23
over our services perhaps this seems a bit complicated let's simplify it with a
5:25
bit complicated let's simplify it with a
5:25
bit complicated let's simplify it with a real world example imagine our city has
5:28
real world example imagine our city has
5:28
real world example imagine our city has a single Central post office instead of
5:31
a single Central post office instead of
5:31
a single Central post office instead of each person delivering their mail
5:33
each person delivering their mail
5:33
each person delivering their mail directly to the recipient that's the
5:35
directly to the recipient that's the
5:35
directly to the recipient that's the pointto point architecture now we need
5:38
pointto point architecture now we need
5:38
pointto point architecture now we need only to have the address of each
5:40
only to have the address of each
5:40
only to have the address of each recipient and send the mail to the
5:43
recipient and send the mail to the
5:43
recipient and send the mail to the central post office which has complete
5:46
central post office which has complete
5:46
central post office which has complete control over the delivery process the
5:49
control over the delivery process the
5:49
control over the delivery process the centralization ensures a secure process
5:51
centralization ensures a secure process
5:51
centralization ensures a secure process because the post office has invested in
5:54
because the post office has invested in
5:54
because the post office has invested in reducing the risk of thief or loss to a
5:57
reducing the risk of thief or loss to a
5:57
reducing the risk of thief or loss to a minimum additionally the post office can
6:00
minimum additionally the post office can
6:00
minimum additionally the post office can easily Monitor and manage mail through a
6:03
easily Monitor and manage mail through a
6:03
easily Monitor and manage mail through a system that keeps track of every parcel
6:05
system that keeps track of every parcel
6:05
system that keeps track of every parcel sent and received this method is an
6:08
sent and received this method is an
6:08
sent and received this method is an improvement over the previous systems
6:10
improvement over the previous systems
6:10
improvement over the previous systems which was established to expedite
6:12
which was established to expedite
6:13
which was established to expedite expedit mild delivery if necessary the
6:15
expedit mild delivery if necessary the
6:15
expedit mild delivery if necessary the post office will establish new processes
6:17
post office will establish new processes
6:17
post office will establish new processes to ensure delivery without involving the
6:19
to ensure delivery without involving the
6:19
to ensure delivery without involving the sender and the receiver in these issues
6:23
sender and the receiver in these issues
6:23
sender and the receiver in these issues just like the post office the central
6:26
just like the post office the central
6:26
just like the post office the central e eii broker Enterprise application
6:30
e eii broker Enterprise application
6:30
e eii broker Enterprise application integration broker features a central
6:32
integration broker features a central
6:32
integration broker features a central Hub responsible for the routing security
6:36
Hub responsible for the routing security
6:36
Hub responsible for the routing security monitoring and transformation of data to
6:38
monitoring and transformation of data to
6:38
monitoring and transformation of data to ensure it is in format that can be
6:41
ensure it is in format that can be
6:41
ensure it is in format that can be understood by the receiving service like
6:44
understood by the receiving service like
6:45
understood by the receiving service like we see this architecture is great it
6:47
we see this architecture is great it
6:47
we see this architecture is great it solves all of the two of the of the P2P
6:51
solves all of the two of the of the P2P
6:51
solves all of the two of the of the P2P problems but it created a new one we now
6:55
problems but it created a new one we now
6:55
problems but it created a new one we now have a single point of failure if the
6:57
have a single point of failure if the
6:57
have a single point of failure if the central Hub go down all the
6:59
central Hub go down all the
6:59
central Hub go down all the communications will fail and it's really
7:02
communications will fail and it's really
7:02
communications will fail and it's really difficult to communicate with a service
7:04
difficult to communicate with a service
7:04
difficult to communicate with a service that we can't integrate into our
7:07
that we can't integrate into our
7:07
that we can't integrate into our centralized AI broker because it's an
7:09
centralized AI broker because it's an
7:09
centralized AI broker because it's an external one that doesn't belong to our
7:11
external one that doesn't belong to our
7:11
external one that doesn't belong to our company so there is a lot of problems
7:13
company so there is a lot of problems
7:13
company so there is a lot of problems with this architecture so and here comes
7:16
with this architecture so and here comes
7:16
with this architecture so and here comes the Enterprise service pass that what so
7:19
the Enterprise service pass that what so
7:20
the Enterprise service pass that what so which which ASB though is giving back
7:22
which which ASB though is giving back
7:22
which which ASB though is giving back the control of the apps to their owners
7:25
the control of the apps to their owners
7:25
the control of the apps to their owners and not integrating all the apps in a
7:27
and not integrating all the apps in a
7:27
and not integrating all the apps in a central H but giving access only to
7:30
central H but giving access only to
7:30
central H but giving access only to business functionality now our system
7:32
business functionality now our system
7:32
business functionality now our system will be scalable easily the integration
7:35
will be scalable easily the integration
7:35
will be scalable easily the integration here is more easy we will have to deal
7:38
here is more easy we will have to deal
7:38
here is more easy we will have to deal only with business services that will be
7:40
only with business services that will be
7:40
only with business services that will be connected via virtual channels no need
7:43
connected via virtual channels no need
7:43
connected via virtual channels no need for direct connection between services
7:45
for direct connection between services
7:45
for direct connection between services this architecture may seem a little bit
7:47
this architecture may seem a little bit
7:48
this architecture may seem a little bit complex so to simplify that I will stick
7:50
complex so to simplify that I will stick
7:50
complex so to simplify that I will stick on our post office example so instead
7:53
on our post office example so instead
7:53
on our post office example so instead for having one Central centralized post
7:56
for having one Central centralized post
7:56
for having one Central centralized post office imagine that we have multiple
7:58
office imagine that we have multiple
7:58
office imagine that we have multiple small post offices
7:59
small post offices
7:59
small post offices that are connected via Dynamic routers
8:02
that are connected via Dynamic routers
8:02
that are connected via Dynamic routers that can change easily if a route
8:05
that can change easily if a route
8:05
that can change easily if a route doesn't function for example if a post
8:08
doesn't function for example if a post
8:08
doesn't function for example if a post office has a problem the mail is simply
8:10
office has a problem the mail is simply
8:10
office has a problem the mail is simply rerouted through other offices that
8:13
rerouted through other offices that
8:13
rerouted through other offices that means there is no single point through
8:15
means there is no single point through
8:15
means there is no single point through which all mail must pass instead of that
8:18
which all mail must pass instead of that
8:18
which all mail must pass instead of that each post each post office can handle
8:21
each post each post office can handle
8:21
each post each post office can handle mail sort them forward them to their
8:24
mail sort them forward them to their
8:24
mail sort them forward them to their destination to pass them to another post
8:26
destination to pass them to another post
8:26
destination to pass them to another post office that is closer to their
8:28
office that is closer to their
8:28
office that is closer to their destination so there is noing point of
8:29
destination so there is noing point of
8:29
destination so there is noing point of failure the system is resilient it can
8:32
failure the system is resilient it can
8:33
failure the system is resilient it can scale easily all of the all of that
8:35
scale easily all of the all of that
8:35
scale easily all of the all of that because it's decentralized so to
8:37
because it's decentralized so to
8:37
because it's decentralized so to implement these types of architecture
8:39
implement these types of architecture
8:39
implement these types of architecture really to to have
8:41
really to to have
8:41
really to to have resilient in inte integration solution
8:44
resilient in inte integration solution
8:45
resilient in inte integration solution we should have we should use some
8:46
we should have we should use some
8:46
we should have we should use some patterns okay so here in this in this
8:48
patterns okay so here in this in this
8:48
patterns okay so here in this in this talk I will try to highlight some some
8:51
talk I will try to highlight some some
8:52
talk I will try to highlight some some known uh because I I can't have time to
8:54
known uh because I I can't have time to
8:54
known uh because I I can't have time to to highlight all the patterns that we
8:56
to highlight all the patterns that we
8:56
to highlight all the patterns that we have in Enterprise Integration solution
8:58
have in Enterprise Integration solution
8:58
have in Enterprise Integration solution so I will try try to highlight some on
9:01
so I will try try to highlight some on
9:01
so I will try try to highlight some on so let me take a little bit of
9:08
water so I will start with the
9:11
water so I will start with the
9:11
water so I will start with the aggregator pattern okay so in a the
9:15
aggregator pattern okay so in a the
9:15
aggregator pattern okay so in a the aggregator pattern a crucial compon the
9:18
aggregator pattern a crucial compon the
9:18
aggregator pattern a crucial compon the aggregator pattern is a crucial
9:19
aggregator pattern is a crucial
9:20
aggregator pattern is a crucial component in Enterprise Integration the
9:22
component in Enterprise Integration the
9:22
component in Enterprise Integration the aggregator is a stateful filter that
9:25
aggregator is a stateful filter that
9:25
aggregator is a stateful filter that plays a vital role in collecting and
9:27
plays a vital role in collecting and
9:27
plays a vital role in collecting and storing individual message
9:30
storing individual message
9:30
storing individual message until it receives a complete set of
9:32
until it receives a complete set of
9:32
until it receives a complete set of related messages it's primary I received
9:35
related messages it's primary I received
9:35
related messages it's primary I received the it's working good okay great so its
9:39
the it's working good okay great so its
9:39
the it's working good okay great so its primary purpose is to enable the
9:41
primary purpose is to enable the
9:41
primary purpose is to enable the processing of multiple related messages
9:43
processing of multiple related messages
9:43
processing of multiple related messages as a single cohesive units this pattern
9:46
as a single cohesive units this pattern
9:46
as a single cohesive units this pattern is essential when dealing with complex
9:49
is essential when dealing with complex
9:49
is essential when dealing with complex message flows where information arrives
9:52
message flows where information arrives
9:52
message flows where information arrives in fragments or from multiple sources
9:55
in fragments or from multiple sources
9:55
in fragments or from multiple sources the aggregator pattern consists of three
9:58
the aggregator pattern consists of three
9:58
the aggregator pattern consists of three k components correlation which
10:00
k components correlation which
10:00
k components correlation which determines which messages Bel Belong
10:03
determines which messages Bel Belong
10:03
determines which messages Bel Belong Together the completeness condition
10:06
Together the completeness condition
10:06
Together the completeness condition which defines when we are ready to
10:08
which defines when we are ready to
10:09
which defines when we are ready to process the collected messages and the
10:11
process the collected messages and the
10:11
process the collected messages and the aggregation algorithm which specifies
10:14
aggregation algorithm which specifies
10:14
aggregation algorithm which specifies how to combine the received messages
10:16
how to combine the received messages
10:16
how to combine the received messages into a single result understanding these
10:19
into a single result understanding these
10:20
into a single result understanding these components is crucial for implementing
10:21
components is crucial for implementing
10:21
components is crucial for implementing an effective aggregator in your
10:23
an effective aggregator in your
10:23
an effective aggregator in your integration Solutions so let's div let's
10:27
integration Solutions so let's div let's
10:27
integration Solutions so let's div let's dive into how to aggregate pattern
10:30
dive into how to aggregate pattern
10:30
dive into how to aggregate pattern actually works as we can see in this
10:32
actually works as we can see in this
10:32
actually works as we can see in this diagram the process begins with incoming
10:35
diagram the process begins with incoming
10:35
diagram the process begins with incoming messages each message goes through a
10:38
messages each message goes through a
10:38
messages each message goes through a correlation check to determine if it
10:41
correlation check to determine if it
10:41
correlation check to determine if it belongs to an existing aggregate or if
10:44
belongs to an existing aggregate or if
10:44
belongs to an existing aggregate or if it should start a new one if there is a
10:46
it should start a new one if there is a
10:46
it should start a new one if there is a match the message is stored in the
10:49
match the message is stored in the
10:49
match the message is stored in the corresponding aggregate if not a new
10:52
corresponding aggregate if not a new
10:52
corresponding aggregate if not a new aggregate is created after each addition
10:55
aggregate is created after each addition
10:55
aggregate is created after each addition aggregator evaluates the completeness
10:58
aggregator evaluates the completeness
10:58
aggregator evaluates the completeness condition if the condition is met it
11:01
condition if the condition is met it
11:01
condition if the condition is met it means we have all the required messages
11:03
means we have all the required messages
11:03
means we have all the required messages and aggregator can then publish the
11:05
and aggregator can then publish the
11:05
and aggregator can then publish the combined
11:07
combined
11:07
combined message so if the condition isn't meant
11:10
message so if the condition isn't meant
11:10
message so if the condition isn't meant the messages are properly grouped and
11:13
the messages are properly grouped and
11:13
the messages are properly grouped and processed together so if if it's it's
11:16
processed together so if if it's it's
11:16
processed together so if if it's it's not met aggregator continues to wait for
11:18
not met aggregator continues to wait for
11:18
not met aggregator continues to wait for more messages this cycle repeats for
11:20
more messages this cycle repeats for
11:20
more messages this cycle repeats for each incoming messages ensuring that
11:22
each incoming messages ensuring that
11:23
each incoming messages ensuring that related messages are properly grouped
11:25
related messages are properly grouped
11:25
related messages are properly grouped and processed together the beauty of
11:28
and processed together the beauty of
11:28
and processed together the beauty of this pattern lies in its ability to
11:31
this pattern lies in its ability to
11:31
this pattern lies in its ability to handle asynchronous and out of order
11:33
handle asynchronous and out of order
11:34
handle asynchronous and out of order message arrivals making it invaluable in
11:37
message arrivals making it invaluable in
11:37
message arrivals making it invaluable in distributed systems where message timing
11:40
distributed systems where message timing
11:40
distributed systems where message timing can be
11:41
can be
11:41
can be unpredictable so when implementing the
11:44
unpredictable so when implementing the
11:44
unpredictable so when implementing the aggregator pattern choosing the right
11:46
aggregator pattern choosing the right
11:46
aggregator pattern choosing the right aggregation strategy because there is
11:48
aggregation strategy because there is
11:48
aggregation strategy because there is not one is crucial let's explore four
11:51
not one is crucial let's explore four
11:51
not one is crucial let's explore four common strategies first we have the we
11:53
common strategies first we have the we
11:53
common strategies first we have the we for all approach the strategy is ideal
11:55
for all approach the strategy is ideal
11:55
for all approach the strategy is ideal when you need complete information
11:57
when you need complete information
11:57
when you need complete information before processing for example if if an
11:59
before processing for example if if an
11:59
before processing for example if if an order fulfillment system you may wait to
12:01
order fulfillment system you may wait to
12:01
order fulfillment system you may wait to conf the confirmation from all
12:03
conf the confirmation from all
12:03
conf the confirmation from all Department before process for proceeding
12:05
Department before process for proceeding
12:05
Department before process for proceeding the second strategy is Timeout here you
12:07
the second strategy is Timeout here you
12:07
the second strategy is Timeout here you set specific time limit for collecting
12:10
set specific time limit for collecting
12:10
set specific time limit for collecting messages this is useful in scenarios
12:14
messages this is useful in scenarios
12:14
messages this is useful in scenarios where you need to balance completeness
12:16
where you need to balance completeness
12:16
where you need to balance completeness with timeliness such as in biding system
12:20
with timeliness such as in biding system
12:20
with timeliness such as in biding system the third strategy is first best this
12:23
the third strategy is first best this
12:23
the third strategy is first best this approach is all for situations where you
12:26
approach is all for situations where you
12:26
approach is all for situations where you need quick results and sequent
12:29
need quick results and sequent
12:30
need quick results and sequent uh response are less critical
12:33
uh response are less critical
12:33
uh response are less critical okay uh lastly we have the external
12:36
okay uh lastly we have the external
12:36
okay uh lastly we have the external event strategy in this case aggregation
12:38
event strategy in this case aggregation
12:38
event strategy in this case aggregation concludes based on a specific business
12:40
concludes based on a specific business
12:40
concludes based on a specific business event like the end of a trading day in
12:43
event like the end of a trading day in
12:44
event like the end of a trading day in financial systems each of these
12:46
financial systems each of these
12:46
financial systems each of these strategies has its place in the choice
12:48
strategies has its place in the choice
12:48
strategies has its place in the choice depends on your specific business
12:50
depends on your specific business
12:50
depends on your specific business requirements and system constraints okay
12:54
requirements and system constraints okay
12:54
requirements and system constraints okay so implementing the aggregator pattern
12:57
so implementing the aggregator pattern
12:57
so implementing the aggregator pattern requires careful consideration of
13:00
requires careful consideration of
13:00
requires careful consideration of several K factors okay first and
13:04
several K factors okay first and
13:04
several K factors okay first and foremost is State Management since the
13:06
foremost is State Management since the
13:06
foremost is State Management since the agregator needs to store partial
13:08
agregator needs to store partial
13:08
agregator needs to store partial Aggregates until they are complete you
13:11
Aggregates until they are complete you
13:11
Aggregates until they are complete you need an efficient way to manage this
13:13
need an efficient way to manage this
13:13
need an efficient way to manage this state possibly using in memory data
13:15
state possibly using in memory data
13:15
state possibly using in memory data structures or even persistent storage
13:18
structures or even persistent storage
13:18
structures or even persistent storage for reliability next you need a robust
13:22
for reliability next you need a robust
13:22
for reliability next you need a robust correlation mechanism this could be
13:24
correlation mechanism this could be
13:24
correlation mechanism this could be based on message properties content or
13:26
based on message properties content or
13:27
based on message properties content or even custom logic to accurately identify
13:29
even custom logic to accurately identify
13:29
even custom logic to accurately identify related messages error handling is
13:32
related messages error handling is
13:32
related messages error handling is another critical aspect you must decide
13:34
another critical aspect you must decide
13:34
another critical aspect you must decide how to deal with missing or late
13:36
how to deal with missing or late
13:36
how to deal with missing or late messages do you time out send alerts or
13:39
messages do you time out send alerts or
13:39
messages do you time out send alerts or perhaps have a recovery mechanism
13:41
perhaps have a recovery mechanism
13:41
perhaps have a recovery mechanism scalability is also crucial especially
13:44
scalability is also crucial especially
13:44
scalability is also crucial especially in high volume systems your
13:46
in high volume systems your
13:46
in high volume systems your implementation should be able to handle
13:48
implementation should be able to handle
13:48
implementation should be able to handle multiple concurrence aggregations
13:50
multiple concurrence aggregations
13:50
multiple concurrence aggregations efficiently lastly don't forget about
13:52
efficiently lastly don't forget about
13:52
efficiently lastly don't forget about monitoring Implement ways to track the
13:54
monitoring Implement ways to track the
13:54
monitoring Implement ways to track the statu of ongoing aggregations and
13:56
statu of ongoing aggregations and
13:56
statu of ongoing aggregations and measure performance this will help you
13:58
measure performance this will help you
13:58
measure performance this will help you identify battl neck and optimize your
14:01
identify battl neck and optimize your
14:01
identify battl neck and optimize your system by carefully addressing these
14:03
system by carefully addressing these
14:03
system by carefully addressing these considerations you can create a robust
14:05
considerations you can create a robust
14:05
considerations you can create a robust and efficient aggregator that
14:06
and efficient aggregator that
14:06
and efficient aggregator that significantly enhance your integration
14:09
significantly enhance your integration
14:09
significantly enhance your integration capabilities so now we will talk about
14:11
capabilities so now we will talk about
14:11
capabilities so now we will talk about the second integration pattern message
14:13
the second integration pattern message
14:13
the second integration pattern message Channel pattern but don't forget about
14:15
Channel pattern but don't forget about
14:15
Channel pattern but don't forget about the first one because we'll do a small
14:17
the first one because we'll do a small
14:17
the first one because we'll do a small demo of the first one
14:20
demo of the first one
14:20
demo of the first one okay so the message Channel pattern is a
14:24
okay so the message Channel pattern is a
14:24
okay so the message Channel pattern is a fundamental Concept in messaging systems
14:26
fundamental Concept in messaging systems
14:26
fundamental Concept in messaging systems a message channel can be taught of as a
14:29
a message channel can be taught of as a
14:29
a message channel can be taught of as a virtual pipe that connects application
14:31
virtual pipe that connects application
14:31
virtual pipe that connects application with within a messaging
14:34
with within a messaging
14:34
with within a messaging infrastructure its primary purpose is to
14:36
infrastructure its primary purpose is to
14:36
infrastructure its primary purpose is to enable reliable asynchronous
14:38
enable reliable asynchronous
14:38
enable reliable asynchronous communication between different parts of
14:40
communication between different parts of
14:40
communication between different parts of a distributed system think of it as the
14:43
a distributed system think of it as the
14:43
a distributed system think of it as the postal service for your applications it
14:45
postal service for your applications it
14:46
postal service for your applications it ensures that messages are delivered to
14:47
ensures that messages are delivered to
14:47
ensures that messages are delivered to the right recipient in the right order
14:50
the right recipient in the right order
14:50
the right recipient in the right order and with the assurance that they will
14:52
and with the assurance that they will
14:52
and with the assurance that they will not be lost along the way there are two
14:55
not be lost along the way there are two
14:55
not be lost along the way there are two main types of message channels
14:57
main types of message channels
14:57
main types of message channels point-to-point channels which Ensure
14:59
point-to-point channels which Ensure
14:59
point-to-point channels which Ensure that only one receiver consume any given
15:01
that only one receiver consume any given
15:01
that only one receiver consume any given message and publish subscribe channels
15:04
message and publish subscribe channels
15:04
message and publish subscribe channels which allow multiple subscribers to
15:05
which allow multiple subscribers to
15:05
which allow multiple subscribers to receive the same message understanding
15:07
receive the same message understanding
15:07
receive the same message understanding these channels types is crucial for
15:10
these channels types is crucial for
15:10
these channels types is crucial for Designing effective messaging
15:12
Designing effective messaging
15:12
Designing effective messaging architectures that meet your specific
15:14
architectures that meet your specific
15:14
architectures that meet your specific communication needs okay so let's break
15:16
communication needs okay so let's break
15:16
communication needs okay so let's break down the anatomy of a message channel to
15:19
down the anatomy of a message channel to
15:19
down the anatomy of a message channel to understand how it functions as we can
15:21
understand how it functions as we can
15:21
understand how it functions as we can see in this diagram the process begins
15:23
see in this diagram the process begins
15:23
see in this diagram the process begins with a cender application this could be
15:26
with a cender application this could be
15:26
with a cender application this could be any part of your system that needs
15:29
any part of your system that needs
15:29
any part of your system that needs communicat information the sender sends
15:31
communicat information the sender sends
15:31
communicat information the sender sends a message to the messaging channel the
15:34
a message to the messaging channel the
15:34
a message to the messaging channel the channel then stores this message in a
15:35
channel then stores this message in a
15:35
channel then stores this message in a queue or a topic depending on whether
15:38
queue or a topic depending on whether
15:38
queue or a topic depending on whether it's pointto point or publish subscribe
15:40
it's pointto point or publish subscribe
15:40
it's pointto point or publish subscribe channel the message wait here until
15:43
channel the message wait here until
15:43
channel the message wait here until receiver is ready to to process it when
15:47
receiver is ready to to process it when
15:47
receiver is ready to to process it when ready the receivers relieve the message
15:50
ready the receivers relieve the message
15:50
ready the receivers relieve the message from the channel throughout this process
15:52
from the channel throughout this process
15:52
from the channel throughout this process is through this process a channel
15:54
is through this process a channel
15:54
is through this process a channel manager oversees the operation handling
15:56
manager oversees the operation handling
15:56
manager oversees the operation handling tasks like creating channels managing
15:59
tasks like creating channels managing
15:59
tasks like creating channels managing their their life cycle and ensuring
16:01
their their life cycle and ensuring
16:01
their their life cycle and ensuring message delivery this structure allow
16:03
message delivery this structure allow
16:03
message delivery this structure allow for loose coupling between senders and
16:06
for loose coupling between senders and
16:06
for loose coupling between senders and receivers as they don't need to be aware
16:08
receivers as they don't need to be aware
16:08
receivers as they don't need to be aware of each other's existence or be
16:11
of each other's existence or be
16:11
of each other's existence or be available at the same same time it's
16:13
available at the same same time it's
16:13
available at the same same time it's this deine that makes message channels
16:16
this deine that makes message channels
16:16
this deine that makes message channels so powerful for building flexible
16:18
so powerful for building flexible
16:18
so powerful for building flexible scalable systems so now that we
16:21
scalable systems so now that we
16:21
scalable systems so now that we understand what message channels are and
16:23
understand what message channels are and
16:23
understand what message channels are and how they work let's explore why you may
16:27
how they work let's explore why you may
16:27
how they work let's explore why you may want to use them in your system
16:29
want to use them in your system
16:29
want to use them in your system one of the primary benefits is the
16:31
one of the primary benefits is the
16:31
one of the primary benefits is the decoupling of components by
16:33
decoupling of components by
16:33
decoupling of components by communicating through channels your
16:36
communicating through channels your
16:36
communicating through channels your applications don't need to know about
16:38
applications don't need to know about
16:38
applications don't need to know about each other directly making your system
16:40
each other directly making your system
16:40
each other directly making your system more modular and easier to maintain a
16:43
more modular and easier to maintain a
16:43
more modular and easier to maintain a synchronous communication is another key
16:45
synchronous communication is another key
16:45
synchronous communication is another key Adventure senders can fire and forget
16:48
Adventure senders can fire and forget
16:48
Adventure senders can fire and forget continuing their work without waiting
16:50
continuing their work without waiting
16:50
continuing their work without waiting for a response which can significantly
16:53
for a response which can significantly
16:53
for a response which can significantly improve your improve system
16:55
improve your improve system
16:55
improve your improve system responsiveness message channels also
16:58
responsiveness message channels also
16:58
responsiveness message channels also enhance scal ility and full tolerance
17:00
enhance scal ility and full tolerance
17:00
enhance scal ility and full tolerance you can easily add new receivers to
17:02
you can easily add new receivers to
17:02
you can easily add new receivers to handle increased load and if a receiver
17:05
handle increased load and if a receiver
17:05
handle increased load and if a receiver fails message can be a delivered when it
17:08
fails message can be a delivered when it
17:08
fails message can be a delivered when it recovers many messages systems also
17:10
recovers many messages systems also
17:10
recovers many messages systems also offer persistence and guaranteed
17:13
offer persistence and guaranteed
17:13
offer persistence and guaranteed delivery ensuring that messages aren't
17:15
delivery ensuring that messages aren't
17:15
delivery ensuring that messages aren't lost even in the phe of system
17:18
lost even in the phe of system
17:18
lost even in the phe of system failures as for use cases message
17:21
failures as for use cases message
17:21
failures as for use cases message channels excel in EX in distributed
17:25
channels excel in EX in distributed
17:25
channels excel in EX in distributed system integration forming the backbone
17:27
system integration forming the backbone
17:27
system integration forming the backbone of even driving architecture
17:29
of even driving architecture
17:29
of even driving architecture they are great for load balancing and
17:31
they are great for load balancing and
17:32
they are great for load balancing and work distribution allowing you to spread
17:34
work distribution allowing you to spread
17:34
work distribution allowing you to spread tasks across multiple workers lastly
17:37
tasks across multiple workers lastly
17:37
tasks across multiple workers lastly they can act as a buffer smoothing out
17:39
they can act as a buffer smoothing out
17:39
they can act as a buffer smoothing out spikes and processing and optimizing
17:42
spikes and processing and optimizing
17:42
spikes and processing and optimizing overall system
17:43
overall system
17:43
overall system performance so implementing message
17:46
performance so implementing message
17:46
performance so implementing message channels in your system requires careful
17:49
channels in your system requires careful
17:49
channels in your system requires careful consideration of several factors first
17:52
consideration of several factors first
17:52
consideration of several factors first think about your channel naming and
17:54
think about your channel naming and
17:54
think about your channel naming and organization strategy a clear
17:56
organization strategy a clear
17:56
organization strategy a clear hierarchical naming convention can make
17:59
hierarchical naming convention can make
17:59
hierarchical naming convention can make your system much easier to understand
18:01
your system much easier to understand
18:01
your system much easier to understand and manage as it grows okay security is
18:04
and manage as it grows okay security is
18:04
and manage as it grows okay security is another crucial aspect Implement proper
18:07
another crucial aspect Implement proper
18:07
another crucial aspect Implement proper access controls to controls to ensure
18:11
access controls to controls to ensure
18:11
access controls to controls to ensure that only authorized application can
18:13
that only authorized application can
18:13
that only authorized application can send and receive on specific channels
18:15
send and receive on specific channels
18:15
send and receive on specific channels message routin and filtering are also
18:19
message routin and filtering are also
18:19
message routin and filtering are also important consider how you will direct
18:21
important consider how you will direct
18:21
important consider how you will direct messages to the right channels and how
18:24
messages to the right channels and how
18:24
messages to the right channels and how receivers can filter incoming messages
18:26
receivers can filter incoming messages
18:27
receivers can filter incoming messages to process only what's relevant to them
18:30
to process only what's relevant to them
18:30
to process only what's relevant to them don't forget about monitoring and
18:31
don't forget about monitoring and
18:31
don't forget about monitoring and management Implement tools to try
18:34
management Implement tools to try
18:34
management Implement tools to try performance message volumes and
18:36
performance message volumes and
18:36
performance message volumes and potential button next lastly pay
18:38
potential button next lastly pay
18:38
potential button next lastly pay attention to Performance Tuning this
18:40
attention to Performance Tuning this
18:40
attention to Performance Tuning this might involve adjusting channels power
18:41
might involve adjusting channels power
18:41
might involve adjusting channels power size implementing message compression or
18:43
size implementing message compression or
18:44
size implementing message compression or fine tuning or message middleware
18:45
fine tuning or message middleware
18:45
fine tuning or message middleware configuration by addressing this
18:46
configuration by addressing this
18:46
configuration by addressing this consideration you can create a robust
18:48
consideration you can create a robust
18:48
consideration you can create a robust messaging infrastructure that serves as
18:51
messaging infrastructure that serves as
18:51
messaging infrastructure that serves as avilable communication backbone before
18:55
avilable communication backbone before
18:55
avilable communication backbone before backbone for your entire system so now
18:58
backbone for your entire system so now
18:58
backbone for your entire system so now we will talk about publish subscribe
19:00
we will talk about publish subscribe
19:00
we will talk about publish subscribe Channel pattern publish subscribe
19:02
Channel pattern publish subscribe
19:02
Channel pattern publish subscribe Channel pattern a powerful tool for
19:03
Channel pattern a powerful tool for
19:03
Channel pattern a powerful tool for evage broadcasting in distributed
19:06
evage broadcasting in distributed
19:06
evage broadcasting in distributed systems at its score a published
19:08
systems at its score a published
19:08
systems at its score a published subscriber Channel often called pubs up
19:10
subscriber Channel often called pubs up
19:10
subscriber Channel often called pubs up channel is a special type of messaging
19:12
channel is a special type of messaging
19:12
channel is a special type of messaging channel that deliver a copy of each
19:15
channel that deliver a copy of each
19:15
channel that deliver a copy of each message to multiple receivers its
19:17
message to multiple receivers its
19:17
message to multiple receivers its primary purpose is to efficiently
19:20
primary purpose is to efficiently
19:20
primary purpose is to efficiently distribute events or information to
19:22
distribute events or information to
19:22
distribute events or information to multiple interested parties
19:24
multiple interested parties
19:24
multiple interested parties simultaneously okay so the key concept
19:27
simultaneously okay so the key concept
19:27
simultaneously okay so the key concept here is the deck coupling of Publishers
19:30
here is the deck coupling of Publishers
19:30
here is the deck coupling of Publishers from the Publishers from subscribers
19:33
from the Publishers from subscribers
19:33
from the Publishers from subscribers Publishers send messages to a topic
19:35
Publishers send messages to a topic
19:35
Publishers send messages to a topic without knowing who will Who will
19:38
without knowing who will Who will
19:38
without knowing who will Who will receive them while subscribers express
19:41
receive them while subscribers express
19:41
receive them while subscribers express interest in one or more topics and
19:44
interest in one or more topics and
19:44
interest in one or more topics and receive messages from those topics
19:47
receive messages from those topics
19:47
receive messages from those topics without knowledge of who published them
19:50
without knowledge of who published them
19:50
without knowledge of who published them this deine makes pops up channels
19:52
this deine makes pops up channels
19:52
this deine makes pops up channels incredibly flexible and scalable
19:55
incredibly flexible and scalable
19:55
incredibly flexible and scalable allowing for Dynamic Addition or removal
19:58
allowing for Dynamic Addition or removal
19:58
allowing for Dynamic Addition or removal of subscri without affecting the
20:00
of subscri without affecting the
20:00
of subscri without affecting the Publishers or other subscribers so let's
20:03
Publishers or other subscribers so let's
20:03
Publishers or other subscribers so let's dive into how publish subscribe Channel
20:05
dive into how publish subscribe Channel
20:05
dive into how publish subscribe Channel actually work as we can see in this
20:07
actually work as we can see in this
20:07
actually work as we can see in this diagram the process begins with
20:10
diagram the process begins with
20:10
diagram the process begins with Publisher this could be any component in
20:12
Publisher this could be any component in
20:12
Publisher this could be any component in your system that has information to
20:14
your system that has information to
20:14
your system that has information to chair the publisher sends an event or
20:17
chair the publisher sends an event or
20:17
chair the publisher sends an event or message to a topic think of topic as a
20:20
message to a topic think of topic as a
20:20
message to a topic think of topic as a broadcasting station once the message
20:22
broadcasting station once the message
20:22
broadcasting station once the message reaches the topic the magic happens the
20:25
reaches the topic the magic happens the
20:25
reaches the topic the magic happens the pops up Channel creates a copy of the
20:27
pops up Channel creates a copy of the
20:27
pops up Channel creates a copy of the message for each Q subscriber and
20:29
message for each Q subscriber and
20:30
message for each Q subscriber and delivers these copy simultaneously in
20:32
delivers these copy simultaneously in
20:32
delivers these copy simultaneously in our diagram we see three subscribers
20:35
our diagram we see three subscribers
20:35
our diagram we see three subscribers each receiving a copy of the message an
20:37
each receiving a copy of the message an
20:37
each receiving a copy of the message an important feature to note is the ability
20:40
important feature to note is the ability
20:40
important feature to note is the ability to dynamically add new subscriber sub
20:43
to dynamically add new subscriber sub
20:43
to dynamically add new subscriber sub subscribers as shown by the dotted line
20:45
subscribers as shown by the dotted line
20:45
subscribers as shown by the dotted line a new subscriber can join at any time
20:48
a new subscriber can join at any time
20:48
a new subscriber can join at any time and will start receiving messages from
20:50
and will start receiving messages from
20:50
and will start receiving messages from that point forward this Dynamic nature
20:52
that point forward this Dynamic nature
20:52
that point forward this Dynamic nature is what make makes pop sub channels so
20:55
is what make makes pop sub channels so
20:55
is what make makes pop sub channels so flexible and Powerful for building every
20:57
flexible and Powerful for building every
20:57
flexible and Powerful for building every driving architect stres
21:00
driving architect stres
21:00
driving architect stres so I will drink some other
21:05
please so the published subscribe
21:08
please so the published subscribe
21:08
please so the published subscribe Channel pattern offers several
21:10
Channel pattern offers several
21:10
Channel pattern offers several significant advantages that make it
21:13
significant advantages that make it
21:13
significant advantages that make it easier a go-to solution for many
21:15
easier a go-to solution for many
21:15
easier a go-to solution for many scenarios one of the primary benefits is
21:18
scenarios one of the primary benefits is
21:18
scenarios one of the primary benefits is the lose decoupling it creates between
21:20
the lose decoupling it creates between
21:20
the lose decoupling it creates between Publishers and subscribers this means
21:23
Publishers and subscribers this means
21:23
Publishers and subscribers this means you can modify either side without
21:25
you can modify either side without
21:25
you can modify either side without affecting the other leading to more
21:27
affecting the other leading to more
21:27
affecting the other leading to more maintainable and flexible systems
21:30
maintainable and flexible systems
21:30
maintainable and flexible systems scalability is another key Advantage you
21:33
scalability is another key Advantage you
21:33
scalability is another key Advantage you can easily add new subscribers to handle
21:37
can easily add new subscribers to handle
21:37
can easily add new subscribers to handle increased to handle increased load or
21:39
increased to handle increased load or
21:39
increased to handle increased load or new requirements without any changes to
21:42
new requirements without any changes to
21:42
new requirements without any changes to the publer flexibility is also strong
21:45
the publer flexibility is also strong
21:45
the publer flexibility is also strong point as subscribers can often filter
21:48
point as subscribers can often filter
21:48
point as subscribers can often filter messages based on content or metadata on
21:51
messages based on content or metadata on
21:51
messages based on content or metadata on Shing they only process relevant
21:53
Shing they only process relevant
21:53
Shing they only process relevant information additionally pop sub
21:56
information additionally pop sub
21:56
information additionally pop sub channels excel at Real Time Event
21:58
channels excel at Real Time Event
21:58
channels excel at Real Time Event propagation making them ideal for
22:02
propagation making them ideal for
22:02
propagation making them ideal for systems that need to react quickly to
22:04
systems that need to react quickly to
22:04
systems that need to react quickly to changing conditions as for use cases Pub
22:07
changing conditions as for use cases Pub
22:07
changing conditions as for use cases Pub sub channels are perfect for news feed
22:11
sub channels are perfect for news feed
22:11
sub channels are perfect for news feed systems where multiple users need to
22:13
systems where multiple users need to
22:14
systems where multiple users need to receive updates simultaneously they are
22:16
receive updates simultaneously they are
22:16
receive updates simultaneously they are extensively used in stock market data
22:18
extensively used in stock market data
22:18
extensively used in stock market data distribution allowing multiple traders
22:20
distribution allowing multiple traders
22:20
distribution allowing multiple traders to receive for example realtime price
22:22
to receive for example realtime price
22:22
to receive for example realtime price updates in The Internet of Things domain
22:25
updates in The Internet of Things domain
22:25
updates in The Internet of Things domain Pub sub channels efficiently broadcast
22:27
Pub sub channels efficiently broadcast
22:27
Pub sub channels efficiently broadcast sensor data to multiple monitoring and
22:30
sensor data to multiple monitoring and
22:30
sensor data to multiple monitoring and Analysis systems lastly there are key
22:32
Analysis systems lastly there are key
22:32
Analysis systems lastly there are key components in many realtime
22:34
components in many realtime
22:34
components in many realtime collaboration tools enabling insted
22:36
collaboration tools enabling insted
22:36
collaboration tools enabling insted updates across multiple clients so
22:39
updates across multiple clients so
22:39
updates across multiple clients so implementing the publish subscribe
22:40
implementing the publish subscribe
22:40
implementing the publish subscribe channels in your system requires careful
22:44
channels in your system requires careful
22:44
channels in your system requires careful consideration of several key factors
22:47
consideration of several key factors
22:47
consideration of several key factors okay first think about your message
22:49
okay first think about your message
22:49
okay first think about your message delivery guarantees do you need at least
22:52
delivery guarantees do you need at least
22:52
delivery guarantees do you need at least once delivery or is it okay if some
22:55
once delivery or is it okay if some
22:55
once delivery or is it okay if some messages are occasionally missed this
22:57
messages are occasionally missed this
22:57
messages are occasionally missed this decision will impact your choice of
23:00
decision will impact your choice of
23:00
decision will impact your choice of messaging middleware and how you handle
23:03
messaging middleware and how you handle
23:03
messaging middleware and how you handle edge cases subscribed management is
23:05
edge cases subscribed management is
23:05
edge cases subscribed management is another crucial aspect consider how
23:07
another crucial aspect consider how
23:07
another crucial aspect consider how subscribers will will register and
23:10
subscribers will will register and
23:10
subscribers will will register and unregister from topics and how you will
23:13
unregister from topics and how you will
23:13
unregister from topics and how you will handle inactive or slow subscribers
23:15
handle inactive or slow subscribers
23:15
handle inactive or slow subscribers topic naming and hierarchy is important
23:18
topic naming and hierarchy is important
23:18
topic naming and hierarchy is important for organizing your event space a
23:21
for organizing your event space a
23:21
for organizing your event space a well-designed topic structure can make
23:25
well-designed topic structure can make
23:25
well-designed topic structure can make it can make it easier to manage and
23:27
it can make it easier to manage and
23:27
it can make it easier to manage and scale your system as it grows
23:29
scale your system as it grows
23:29
scale your system as it grows message filtering is another
23:30
message filtering is another
23:30
message filtering is another consideration decide whether filtering
23:33
consideration decide whether filtering
23:33
consideration decide whether filtering should happen at the channel level or
23:35
should happen at the channel level or
23:35
should happen at the channel level or the Subscribe level and how complex your
23:37
the Subscribe level and how complex your
23:37
the Subscribe level and how complex your filtering needs are lastly pay close
23:41
filtering needs are lastly pay close
23:41
filtering needs are lastly pay close attention to Performance and scalability
23:43
attention to Performance and scalability
23:43
attention to Performance and scalability as the number of Publishers subscribers
23:45
as the number of Publishers subscribers
23:45
as the number of Publishers subscribers and message messages grows you may have
23:48
and message messages grows you may have
23:48
and message messages grows you may have you may have need to implement
23:49
you may have need to implement
23:49
you may have need to implement strategies like messaging patching load
23:52
strategies like messaging patching load
23:52
strategies like messaging patching load balancing or even custom optimization in
23:54
balancing or even custom optimization in
23:54
balancing or even custom optimization in your message infrastructure by
23:56
your message infrastructure by
23:56
your message infrastructure by addressing this consideration you you
23:58
addressing this consideration you you
23:58
addressing this consideration you you can create a robust pubsub system that
24:01
can create a robust pubsub system that
24:01
can create a robust pubsub system that efficiently handles your event
24:03
efficiently handles your event
24:03
efficiently handles your event distribution needs so now we'll talk
24:06
distribution needs so now we'll talk
24:06
distribution needs so now we'll talk about process manager pattern so the
24:09
about process manager pattern so the
24:09
about process manager pattern so the process manager pattern a powerful tool
24:11
process manager pattern a powerful tool
24:11
process manager pattern a powerful tool for orchestrating complex workflows in
24:14
for orchestrating complex workflows in
24:14
for orchestrating complex workflows in distributed systems at its core a
24:17
distributed systems at its core a
24:17
distributed systems at its core a process manager process manager is a
24:19
process manager process manager is a
24:19
process manager process manager is a central component that maintains the
24:21
central component that maintains the
24:21
central component that maintains the state of sequence of steps it determines
24:24
state of sequence of steps it determines
24:24
state of sequence of steps it determines the next processing step based on
24:26
the next processing step based on
24:26
the next processing step based on intermediate results it's purpose is to
24:29
intermediate results it's purpose is to
24:29
intermediate results it's purpose is to manage and coordinate complex M multipe
24:32
manage and coordinate complex M multipe
24:32
manage and coordinate complex M multipe business processes that may involve
24:34
business processes that may involve
24:34
business processes that may involve multiple services or applications the
24:36
multiple services or applications the
24:36
multiple services or applications the key concept here is centralized control
24:38
key concept here is centralized control
24:38
key concept here is centralized control of distributor process unlike simpler
24:42
of distributor process unlike simpler
24:42
of distributor process unlike simpler patterns that may use predefined
24:43
patterns that may use predefined
24:43
patterns that may use predefined sequence of stepes the process manager
24:45
sequence of stepes the process manager
24:45
sequence of stepes the process manager can make decisions on the Fly adapting
24:48
can make decisions on the Fly adapting
24:48
can make decisions on the Fly adapting the process flow based on the results of
24:51
the process flow based on the results of
24:51
the process flow based on the results of each step this makes it incredibly
24:53
each step this makes it incredibly
24:53
each step this makes it incredibly flexible and capable of handling complex
24:56
flexible and capable of handling complex
24:56
flexible and capable of handling complex business logic that might be difficult
24:58
business logic that might be difficult
24:58
business logic that might be difficult to complement with other pattern let's
25:00
to complement with other pattern let's
25:00
to complement with other pattern let's dive into how the proc manager actually
25:01
dive into how the proc manager actually
25:02
dive into how the proc manager actually works as we can see in the diagram the
25:04
works as we can see in the diagram the
25:04
works as we can see in the diagram the process begins with a trigger message
25:06
process begins with a trigger message
25:06
process begins with a trigger message this could be a customer order a system
25:08
this could be a customer order a system
25:08
this could be a customer order a system event or any other thing that kicks
25:11
event or any other thing that kicks
25:11
event or any other thing that kicks business process the trigger message is
25:13
business process the trigger message is
25:13
business process the trigger message is received by the process manager which
25:16
received by the process manager which
25:16
received by the process manager which then makes a decision about what step to
25:18
then makes a decision about what step to
25:18
then makes a decision about what step to execute first the decision is based on
25:21
execute first the decision is based on
25:21
execute first the decision is based on predefined rules and the concept of the
25:23
predefined rules and the concept of the
25:23
predefined rules and the concept of the trigger message the process manager then
25:25
trigger message the process manager then
25:25
trigger message the process manager then initiates the chosen step which could be
25:28
initiates the chosen step which could be
25:28
initiates the chosen step which could be a call to a microservice a database
25:30
a call to a microservice a database
25:30
a call to a microservice a database operation or any other tool and any
25:33
operation or any other tool and any
25:33
operation or any other tool and any other action once that step completes it
25:36
other action once that step completes it
25:36
other action once that step completes it returns a result to the process manager
25:38
returns a result to the process manager
25:38
returns a result to the process manager the process manager then uses this
25:40
the process manager then uses this
25:40
the process manager then uses this results along with its internal state to
25:43
results along with its internal state to
25:43
results along with its internal state to decide on the next step the cycle
25:46
decide on the next step the cycle
25:46
decide on the next step the cycle continues with the process manager
25:48
continues with the process manager
25:48
continues with the process manager orating each step until the entire
25:51
orating each step until the entire
25:51
orating each step until the entire process is complete at that point the
25:54
process is complete at that point the
25:54
process is complete at that point the process manager produces the final
25:55
process manager produces the final
25:55
process manager produces the final output decentralized decision making all
25:58
output decentralized decision making all
25:58
output decentralized decision making all for complex conditional workflows that
26:01
for complex conditional workflows that
26:01
for complex conditional workflows that can adapt based on the results of each
26:04
can adapt based on the results of each
26:04
can adapt based on the results of each step so the process manager pattern
26:08
step so the process manager pattern
26:08
step so the process manager pattern offers several significant advantages
26:10
offers several significant advantages
26:10
offers several significant advantages that make it invaluable for complex
26:13
that make it invaluable for complex
26:13
that make it invaluable for complex business processes one of the primary
26:16
business processes one of the primary
26:16
business processes one of the primary benefits is the centralized control it
26:18
benefits is the centralized control it
26:18
benefits is the centralized control it provides over complex workflows
26:21
provides over complex workflows
26:21
provides over complex workflows centralization makes it easier to
26:23
centralization makes it easier to
26:23
centralization makes it easier to understand Monitor and modify business
26:26
understand Monitor and modify business
26:26
understand Monitor and modify business processes another key Advantage is its
26:28
processes another key Advantage is its
26:28
processes another key Advantage is its ability to handle long running stateful
26:31
ability to handle long running stateful
26:31
ability to handle long running stateful processes unlike simpler patterns a
26:33
processes unlike simpler patterns a
26:33
processes unlike simpler patterns a process manager can maintain State over
26:36
process manager can maintain State over
26:36
process manager can maintain State over extended periods making it perfect for
26:38
extended periods making it perfect for
26:38
extended periods making it perfect for processes that may take hours days or
26:41
processes that may take hours days or
26:41
processes that may take hours days or even weeks to complete flexibility is
26:45
even weeks to complete flexibility is
26:45
even weeks to complete flexibility is another strong point the process manager
26:47
another strong point the process manager
26:47
another strong point the process manager can adapt the processes flow based on
26:49
can adapt the processes flow based on
26:49
can adapt the processes flow based on intermediate results allowing for
26:51
intermediate results allowing for
26:51
intermediate results allowing for dynamic condition based workflows
26:53
dynamic condition based workflows
26:53
dynamic condition based workflows additionally it provides improved
26:55
additionally it provides improved
26:55
additionally it provides improved visability and manageability of business
26:57
visability and manageability of business
26:57
visability and manageability of business processes as all the logic is
26:59
processes as all the logic is
26:59
processes as all the logic is centralized in one components as for use
27:02
centralized in one components as for use
27:02
centralized in one components as for use cases process managers excel in
27:05
cases process managers excel in
27:05
cases process managers excel in scenarios like order fulfillments
27:08
scenarios like order fulfillments
27:08
scenarios like order fulfillments systems where multiple steps like
27:10
systems where multiple steps like
27:10
systems where multiple steps like inventory check payment processing and
27:13
inventory check payment processing and
27:13
inventory check payment processing and chipping need to be
27:15
chipping need to be
27:15
chipping need to be coordinated they are also ideal for
27:17
coordinated they are also ideal for
27:17
coordinated they are also ideal for insurance claim processing where the
27:19
insurance claim processing where the
27:19
insurance claim processing where the stepes May Vary based on the type of
27:21
stepes May Vary based on the type of
27:21
stepes May Vary based on the type of claim and its complexity Trav book
27:24
claim and its complexity Trav book
27:24
claim and its complexity Trav book booking systems often uses process
27:26
booking systems often uses process
27:26
booking systems often uses process manager to coordinate flights or hotels
27:28
manager to coordinate flights or hotels
27:28
manager to coordinate flights or hotels and car rentals lastly they are perfect
27:31
and car rentals lastly they are perfect
27:31
and car rentals lastly they are perfect for M stage approval workflow where
27:34
for M stage approval workflow where
27:34
for M stage approval workflow where different approvals may be needed based
27:36
different approvals may be needed based
27:36
different approvals may be needed based on various condition so implementing a
27:40
on various condition so implementing a
27:40
on various condition so implementing a process manager in your system requires
27:42
process manager in your system requires
27:42
process manager in your system requires careful consideration of several K
27:44
careful consideration of several K
27:44
careful consideration of several K Factor first and foremost is stateful
27:48
Factor first and foremost is stateful
27:48
Factor first and foremost is stateful State Management you need to decide how
27:50
State Management you need to decide how
27:50
State Management you need to decide how to persist the state of each process
27:52
to persist the state of each process
27:52
to persist the state of each process instance ensuring that it can be
27:55
instance ensuring that it can be
27:55
instance ensuring that it can be recovered in case of system failures
27:57
recovered in case of system failures
27:57
recovered in case of system failures this often involves using
28:01
this often involves using
28:01
this often involves using database to store process States error
28:03
database to store process States error
28:03
database to store process States error handling and recovering are crucial you
28:06
handling and recovering are crucial you
28:06
handling and recovering are crucial you need to design how your process manager
28:09
need to design how your process manager
28:09
need to design how your process manager will handle failures at various stages
28:12
will handle failures at various stages
28:12
will handle failures at various stages and how it can resume processes from the
28:14
and how it can resume processes from the
28:14
and how it can resume processes from the point of failure scalability and
28:17
point of failure scalability and
28:17
point of failure scalability and performance are important considerations
28:19
performance are important considerations
28:19
performance are important considerations especially for high volume systems you
28:21
especially for high volume systems you
28:21
especially for high volume systems you may need to implement strategies like
28:24
may need to implement strategies like
28:24
may need to implement strategies like like uh charting or load balancing to
28:27
like uh charting or load balancing to
28:27
like uh charting or load balancing to distribute the load across multiple
28:29
distribute the load across multiple
28:29
distribute the load across multiple process manager instances monitoring and
28:32
process manager instances monitoring and
28:32
process manager instances monitoring and logging are essential for maintaining
28:33
logging are essential for maintaining
28:33
logging are essential for maintaining maintaining and troubleshooting complex
28:35
maintaining and troubleshooting complex
28:35
maintaining and troubleshooting complex workf flows implementing comprehensive
28:38
workf flows implementing comprehensive
28:38
workf flows implementing comprehensive loging of each step and decision made by
28:40
loging of each step and decision made by
28:40
loging of each step and decision made by the process
28:42
the process
28:42
the process manager lastly consider how you will
28:45
manager lastly consider how you will
28:45
manager lastly consider how you will Define in uh Define in verion your
28:48
Define in uh Define in verion your
28:48
Define in uh Define in verion your processes many implementation use a
28:51
processes many implementation use a
28:51
processes many implementation use a domain specific language or visual tools
28:53
domain specific language or visual tools
28:53
domain specific language or visual tools to Define processes and you will need a
28:54
to Define processes and you will need a
28:54
to Define processes and you will need a strategy strategy for versioning these
28:57
strategy strategy for versioning these
28:57
strategy strategy for versioning these Define definitions as you your business
29:00
Define definitions as you your business
29:00
Define definitions as you your business processes evolve over time by addressing
29:02
processes evolve over time by addressing
29:02
processes evolve over time by addressing these considerations you can create a
29:04
these considerations you can create a
29:04
these considerations you can create a robust process manager that effectively
29:07
robust process manager that effectively
29:07
robust process manager that effectively orchestrates your complex business
29:09
orchestrates your complex business
29:09
orchestrates your complex business workflows
29:11
workflows
29:11
workflows so now we we we have we have studied
29:14
so now we we we have we have studied
29:14
so now we we we have we have studied four integration patterns I told you
29:17
four integration patterns I told you
29:17
four integration patterns I told you that you don't forget about the
29:19
that you don't forget about the
29:19
that you don't forget about the aggregator one so here I will try to
29:21
aggregator one so here I will try to
29:21
aggregator one so here I will try to give you an example of Enterprise
29:23
give you an example of Enterprise
29:23
give you an example of Enterprise Integration solution that exist in the
29:25
Integration solution that exist in the
29:25
Integration solution that exist in the market there is for example apash camel
29:27
market there is for example apash camel
29:27
market there is for example apash camel there is muft and there's a lot of other
29:30
there is muft and there's a lot of other
29:30
there is muft and there's a lot of other solutions that you can use
29:33
solutions that you can use
29:33
solutions that you can use you I I there's large Solutions so you
29:35
you I I there's large Solutions so you
29:35
you I I there's large Solutions so you don't need to code all of all of this
29:38
don't need to code all of all of this
29:38
don't need to code all of all of this integration patterns maybe for if you
29:40
integration patterns maybe for if you
29:40
integration patterns maybe for if you have more complex use cases but there is
29:43
have more complex use cases but there is
29:43
have more complex use cases but there is tools that do that for us for example
29:46
tools that do that for us for example
29:46
tools that do that for us for example here we will try to to understand how mu
29:49
here we will try to to understand how mu
29:49
here we will try to to understand how mu soft the solution for for for for
29:53
soft the solution for for for for
29:53
soft the solution for for for for integration Solutions can are
29:56
integration Solutions can are
29:56
integration Solutions can are implementing the aggregate
29:58
implementing the aggregate
29:58
implementing the aggregate we will see how they implemented the
30:00
we will see how they implemented the
30:00
we will see how they implemented the aggregator so here we can see diagram
30:02
aggregator so here we can see diagram
30:02
aggregator so here we can see diagram now we will try to dive into mules of
30:06
now we will try to dive into mules of
30:06
now we will try to dive into mules of time based aggregator here we will only
30:09
time based aggregator here we will only
30:09
time based aggregator here we will only see the aggregator with the time based
30:11
see the aggregator with the time based
30:11
see the aggregator with the time based that will take for example that will
30:13
that will take for example that will
30:13
that will take for example that will aggregate results every five minutes
30:15
aggregate results every five minutes
30:15
aggregate results every five minutes okay so and we'll see how it's
30:18
okay so and we'll see how it's
30:18
okay so and we'll see how it's implemented let's start with our new
30:20
implemented let's start with our new
30:20
implemented let's start with our new flow this is the first class imagine
30:22
flow this is the first class imagine
30:22
flow this is the first class imagine orders following into our systems order
30:25
orders following into our systems order
30:25
orders following into our systems order for five shirts order for three pants
30:27
for five shirts order for three pants
30:27
for five shirts order for three pants and so on
30:28
and so on
30:28
and so on as these orders arrive the mule flow
30:31
as these orders arrive the mule flow
30:31
as these orders arrive the mule flow cause the execute cause the execute
30:33
cause the execute cause the execute
30:33
cause the execute cause the execute method on our time based aggregation
30:36
method on our time based aggregation
30:36
method on our time based aggregation operation executor it's a class there
30:39
operation executor it's a class there
30:39
operation executor it's a class there this executor then calls the aggregate
30:42
this executor then calls the aggregate
30:42
this executor then calls the aggregate meod method on the abstract aggregator
30:45
meod method on the abstract aggregator
30:45
meod method on the abstract aggregator executor think of of this as the plane
30:48
executor think of of this as the plane
30:48
executor think of of this as the plane of our operation coordinate coordinating
30:50
of our operation coordinate coordinating
30:50
of our operation coordinate coordinating all the moving Parts first the the
30:53
all the moving Parts first the the
30:53
all the moving Parts first the the abstract aggregate executor reaches out
30:56
abstract aggregate executor reaches out
30:56
abstract aggregate executor reaches out to the object store our persistent
30:58
to the object store our persistent
30:58
to the object store our persistent storage here we are talking about the
31:00
storage here we are talking about the
31:00
storage here we are talking about the State Management our persistent storage
31:03
State Management our persistent storage
31:03
State Management our persistent storage it's like digital faing cabinet ring any
31:06
it's like digital faing cabinet ring any
31:06
it's like digital faing cabinet ring any previous aggregation state if there is
31:08
previous aggregation state if there is
31:08
previous aggregation state if there is if we start already aggregation he will
31:10
if we start already aggregation he will
31:10
if we start already aggregation he will see if there is one he will retrieve it
31:12
see if there is one he will retrieve it
31:12
see if there is one he will retrieve it for example it may recall last time have
31:14
for example it may recall last time have
31:14
for example it may recall last time have 50 orders he will he will take he will
31:16
50 orders he will he will take he will
31:16
50 orders he will he will take he will take he will take the orders that he
31:18
take he will take the orders that he
31:18
take he will take the orders that he will that that he needs next we get our
31:21
will that that he needs next we get our
31:22
will that that he needs next we get our simple aggregated content this is our
31:24
simple aggregated content this is our
31:24
simple aggregated content this is our aggregation basket where we will collect
31:26
aggregation basket where we will collect
31:26
aggregation basket where we will collect all the orders for the dur 5 minutes we
31:28
all the orders for the dur 5 minutes we
31:28
all the orders for the dur 5 minutes we will collect them there now here is
31:30
will collect them there now here is
31:30
will collect them there now here is where the time based magic happens we
31:32
where the time based magic happens we
31:32
where the time based magic happens we will call register async aggregation if
31:34
will call register async aggregation if
31:34
will call register async aggregation if needed it's a method setting up a task
31:37
needed it's a method setting up a task
31:37
needed it's a method setting up a task to run every five minutes it's like
31:39
to run every five minutes it's like
31:39
to run every five minutes it's like setting a kitchen timer for our
31:41
setting a kitchen timer for our
31:41
setting a kitchen timer for our aggregation it it will run for five
31:43
aggregation it it will run for five
31:43
aggregation it it will run for five minutes as each order comes in we call
31:47
minutes as each order comes in we call
31:47
minutes as each order comes in we call add storage on our simple aggregated
31:49
add storage on our simple aggregated
31:49
add storage on our simple aggregated content it's as if we are putting each
31:52
content it's as if we are putting each
31:52
content it's as if we are putting each order slip into our aggregation basket
31:55
order slip into our aggregation basket
31:55
order slip into our aggregation basket okay after processing each order we
31:58
okay after processing each order we
31:58
okay after processing each order we update our shared information in the
32:00
update our shared information in the
32:00
update our shared information in the object store it's like updating our CH
32:03
object store it's like updating our CH
32:03
object store it's like updating our CH change okay so this processes process
32:05
change okay so this processes process
32:05
change okay so this processes process repeats for every order that arrives
32:07
repeats for every order that arrives
32:07
repeats for every order that arrives within the five minutes that we decided
32:10
within the five minutes that we decided
32:10
within the five minutes that we decided now let's fast forward five minutes our
32:12
now let's fast forward five minutes our
32:12
now let's fast forward five minutes our timer goes off it's it's it's finished
32:14
timer goes off it's it's it's finished
32:14
timer goes off it's it's it's finished five minutes so so here on assen
32:18
five minutes so so here on assen
32:18
five minutes so so here on assen aggregation execution is called this is
32:21
aggregation execution is called this is
32:21
aggregation execution is called this is where we process all the orders we've
32:23
where we process all the orders we've
32:23
where we process all the orders we've collected first we call get aggregated
32:25
collected first we call get aggregated
32:25
collected first we call get aggregated elements on our simple aggregated
32:27
elements on our simple aggregated
32:27
elements on our simple aggregated content
32:28
content
32:28
content it's like dumping out our basket okay of
32:31
it's like dumping out our basket okay of
32:31
it's like dumping out our basket okay of of of orders next we check if anyone's
32:34
of of orders next we check if anyone's
32:34
of of orders next we check if anyone's waiting for our results in mule we do
32:36
waiting for our results in mule we do
32:36
waiting for our results in mule we do this by calling get listener in the
32:39
this by calling get listener in the
32:39
this by calling get listener in the aggregation manager will you will see
32:40
aggregation manager will you will see
32:40
aggregation manager will you will see all of that in in in the demo if there
32:43
all of that in in in the demo if there
32:43
all of that in in in the demo if there is a listener we notify it with our
32:46
is a listener we notify it with our
32:46
is a listener we notify it with our aggregated data using notify listener on
32:49
aggregated data using notify listener on
32:49
aggregated data using notify listener on complete it's like calling out in the
32:51
complete it's like calling out in the
32:51
complete it's like calling out in the last 5 minutes we proceed we proceeded
32:54
last 5 minutes we proceed we proceeded
32:54
last 5 minutes we proceed we proceeded 20 orders finally we call resit group
32:56
20 orders finally we call resit group
32:57
20 orders finally we call resit group this clears out our simple aggregated
32:58
this clears out our simple aggregated
32:59
this clears out our simple aggregated content ready for the next 5 minutes
33:01
content ready for the next 5 minutes
33:01
content ready for the next 5 minutes window it's like emptying our basket
33:04
window it's like emptying our basket
33:04
window it's like emptying our basket ready to collect new orders what makes
33:06
ready to collect new orders what makes
33:06
ready to collect new orders what makes this approach powerful is its efficiency
33:08
this approach powerful is its efficiency
33:08
this approach powerful is its efficiency we're not storing every single order for
33:10
we're not storing every single order for
33:10
we're not storing every single order for over instead we are summarizing our data
33:12
over instead we are summarizing our data
33:13
over instead we are summarizing our data every five minutes providing regular
33:15
every five minutes providing regular
33:15
every five minutes providing regular insights into our sales so this this
33:20
insights into our sales so this this
33:20
insights into our sales so this this this pattern is incredible while we've
33:22
this pattern is incredible while we've
33:22
this pattern is incredible while we've talked about order processing you could
33:23
talked about order processing you could
33:23
talked about order processing you could use you could use it for time based data
33:25
use you could use it for time based data
33:25
use you could use it for time based data monitoring website traffic for example
33:27
monitoring website traffic for example
33:27
monitoring website traffic for example count visits every minute tracking stock
33:30
count visits every minute tracking stock
33:30
count visits every minute tracking stock prices aggregate prices you can use it
33:33
prices aggregate prices you can use it
33:33
prices aggregate prices you can use it for for lot for for a lot of use cases
33:36
for for lot for for a lot of use cases
33:36
for for lot for for a lot of use cases so here we will try to see a real demo
33:40
so here we will try to see a real demo
33:40
so here we will try to see a real demo with moft and how we can Implement
33:43
with moft and how we can Implement
33:43
with moft and how we can Implement Implement that very fast with
33:46
Implement that very fast with
33:46
Implement that very fast with moft so here I
33:48
moft so here I
33:48
moft so here I will uh I will share now so how can I
33:53
will uh I will share now so how can I
33:53
will uh I will share now so how can I share that be like
33:56
share that be like
33:56
share that be like that and uhuh okay great so I will share
34:00
that and uhuh okay great so I will share
34:00
that and uhuh okay great so I will share my screen
34:07
now so here I
34:09
now so here I
34:09
now so here I will share my
34:13
will share my
34:13
will share my screen with you so you can see the demo
34:17
screen with you so you can see the demo
34:17
screen with you so you can see the demo okay great so here with with moft I will
34:20
okay great so here with with moft I will
34:20
okay great so here with with moft I will create a flow a flow of this aggregation
34:23
create a flow a flow of this aggregation
34:23
create a flow a flow of this aggregation I will create a new one here because um
34:25
I will create a new one here because um
34:25
I will create a new one here because um I was trying to to prepare the demo for
34:27
I was trying to to prepare the demo for
34:28
I was trying to to prepare the demo for this stock I will name it c conference
34:31
this stock I will name it c conference
34:32
this stock I will name it c conference okay so it's very simple here I will
34:34
okay so it's very simple here I will
34:35
okay so it's very simple here I will call an aggregator I need a Time based
34:38
call an aggregator I need a Time based
34:39
call an aggregator I need a Time based one it's here here what I will try to do
34:42
one it's here here what I will try to do
34:42
one it's here here what I will try to do is I will try to
34:44
is I will try to
34:44
is I will try to log what the aggregator is doing and how
34:48
log what the aggregator is doing and how
34:48
log what the aggregator is doing and how I can call this aggregator I will I will
34:51
I can call this aggregator I will I will
34:51
I can call this aggregator I will I will Define HTTP listener to call the
34:54
Define HTTP listener to call the
34:54
Define HTTP listener to call the aggregator and what we said before when
34:57
aggregator and what we said before when
34:57
aggregator and what we said before when need an aggregator listener for the
35:00
need an aggregator listener for the
35:00
need an aggregator listener for the ending of what the aggregator is doing
35:02
ending of what the aggregator is doing
35:03
ending of what the aggregator is doing and here I will add a simple Transformer
35:07
and here I will add a simple Transformer
35:07
and here I will add a simple Transformer transform our message that we will that
35:09
transform our message that we will that
35:09
transform our message that we will that we will print to show you in the
35:11
we will print to show you in the
35:11
we will print to show you in the logger here it
35:13
logger here it
35:13
logger here it is and here I will also add a
35:17
is and here I will also add a
35:17
is and here I will also add a Transformer so we can see our our login
35:21
Transformer so we can see our our login
35:21
Transformer so we can see our our login okay so now I need to configure my
35:24
okay so now I need to configure my
35:24
okay so now I need to configure my components it's not working now okay so
35:27
components it's not working now okay so
35:27
components it's not working now okay so I will go to The Listener I will choose
35:30
I will go to The Listener I will choose
35:30
I will go to The Listener I will choose the listener it's uh one that I already
35:32
the listener it's uh one that I already
35:32
the listener it's uh one that I already set uped here uh with this
35:35
set uped here uh with this
35:35
set uped here uh with this port okay so I will go to
35:39
port okay so I will go to
35:39
port okay so I will go to the I will go to the time here I will
35:42
the I will go to the time here I will
35:42
the I will go to the time here I will Define the I will Define uh no no it's
35:46
Define the I will Define uh no no it's
35:46
Define the I will Define uh no no it's not here it's here it's that's that's
35:49
not here it's here it's that's that's
35:49
not here it's here it's that's that's what what what we do in in demos I
35:52
what what what we do in in demos I
35:52
what what what we do in in demos I should I should not let that okay so
35:55
should I should not let that okay so
35:55
should I should not let that okay so here I will call the listener
35:58
here I will call the listener
35:58
here I will call the listener that I just
36:02
created here I will call it here and I
36:05
created here I will call it here and I
36:05
created here I will call it here and I will call a pet conference okay this is
36:08
will call a pet conference okay this is
36:09
will call a pet conference okay this is the pet here I will call my aggregator I
36:12
the pet here I will call my aggregator I
36:12
the pet here I will call my aggregator I will name it Time conference okay we
36:16
will name it Time conference okay we
36:16
will name it Time conference okay we should not forget it every 10 seconds he
36:19
should not forget it every 10 seconds he
36:19
should not forget it every 10 seconds he should run every 10 seconds here I will
36:22
should run every 10 seconds here I will
36:22
should run every 10 seconds here I will transform my message to gon to print it
36:26
transform my message to gon to print it
36:26
transform my message to gon to print it so it will be
36:28
so it will be
36:28
so it will be it would be cool here I will call it in
36:31
it would be cool here I will call it in
36:31
it would be cool here I will call it in my
36:33
my
36:33
my logger here I will configure my listener
36:37
logger here I will configure my listener
36:37
logger here I will configure my listener I will tell him listen my listener
36:39
I will tell him listen my listener
36:39
I will tell him listen my listener aggregator listen for this aggregator I
36:42
aggregator listen for this aggregator I
36:42
aggregator listen for this aggregator I copy paste the name and here I will use
36:46
copy paste the name and here I will use
36:46
copy paste the name and here I will use my Transformer to transform the payload
36:49
my Transformer to transform the payload
36:49
my Transformer to transform the payload and here I will print it okay cool so
36:52
and here I will print it okay cool so
36:52
and here I will print it okay cool so now I will try to run my demo hope it
36:56
now I will try to run my demo hope it
36:56
now I will try to run my demo hope it will work
37:00
and what I should do is
37:04
and what I should do is
37:04
and what I should do is that I should call my I name it I name
37:09
that I should call my I name it I name
37:09
that I should call my I name it I name it what conference I think conference
37:11
it what conference I think conference
37:11
it what conference I think conference yes so here I have
37:14
yes so here I have
37:14
yes so here I have my my mu soft running and my flow is
37:18
my my mu soft running and my flow is
37:18
my my mu soft running and my flow is running now I can see it and here I can
37:21
running now I can see it and here I can
37:21
running now I can see it and here I can see how the aggregator is working
37:23
see how the aggregator is working
37:24
see how the aggregator is working so we will wait for the process it's
37:26
so we will wait for the process it's
37:26
so we will wait for the process it's cool now it's working working so I will
37:28
cool now it's working working so I will
37:28
cool now it's working working so I will clear the console so I will get nothing
37:32
clear the console so I will get nothing
37:32
clear the console so I will get nothing so we can show the results here I will
37:35
so we can show the results here I will
37:35
so we can show the results here I will call conference and I will send it some
37:38
call conference and I will send it some
37:38
call conference and I will send it some uh I will say t-shirt and after that I
37:41
uh I will say t-shirt and after that I
37:41
uh I will say t-shirt and after that I will say pants for example nine t-shirts
37:43
will say pants for example nine t-shirts
37:43
will say pants for example nine t-shirts so here when I will look look with me
37:46
so here when I will look look with me
37:46
so here when I will look look with me here you will see something so when I
37:48
here you will see something so when I
37:48
here you will see something so when I will
37:49
will
37:49
will click I will get the T-shirt okay here I
37:52
click I will get the T-shirt okay here I
37:52
click I will get the T-shirt okay here I will tell him I want six pant they
37:56
will tell him I want six pant they
37:56
will tell him I want six pant they started the counting for to 10 seconds
37:58
started the counting for to 10 seconds
37:58
started the counting for to 10 seconds here I have now he Aggregates dos for
38:01
here I have now he Aggregates dos for
38:01
here I have now he Aggregates dos for nine and six okay so here maybe I did
38:07
nine and six okay so here maybe I did
38:07
nine and six okay so here maybe I did something I forget to to I forget the
38:10
something I forget to to I forget the
38:10
something I forget to to I forget the payload okay sorry for that sorry for
38:13
payload okay sorry for that sorry for
38:13
payload okay sorry for that sorry for that I forget to print the final
38:17
that I forget to print the final
38:17
that I forget to print the final page so sorry for that I will rerun my
38:21
page so sorry for that I will rerun my
38:21
page so sorry for that I will rerun my process
38:22
process
38:22
process now so here I will try to show you how
38:25
now so here I will try to show you how
38:25
now so here I will try to show you how how it works
38:29
how it works
38:29
how it works I think I'm running out of time I'll try
38:32
I think I'm running out of time I'll try
38:32
I think I'm running out of time I'll try to do my best to
38:33
to do my best to
38:33
to do my best to finish so here we can you can see with
38:39
finish so here we can you can see with
38:39
finish so here we can you can see with me I will clear my
38:42
me I will clear my
38:42
me I will clear my uh aggregator okay my console is it
38:47
uh aggregator okay my console is it
38:47
uh aggregator okay my console is it working is it finish it yes great so I
38:52
working is it finish it yes great so I
38:52
working is it finish it yes great so I will clear this here I will call
38:55
will clear this here I will call
38:55
will clear this here I will call conference slash Pence six here I will
38:59
conference slash Pence six here I will
38:59
conference slash Pence six here I will add four
39:02
add four
39:02
add four t
39:04
t
39:04
t shirts call it now he aggregated
39:08
shirts call it now he aggregated
39:08
shirts call it now he aggregated 64 and the final state is 64 so to show
39:11
64 and the final state is 64 so to show
39:11
64 and the final state is 64 so to show you how it works maybe I should copy
39:13
you how it works maybe I should copy
39:13
you how it works maybe I should copy paste simple simple total so here we
39:17
paste simple simple total so here we
39:17
paste simple simple total so here we will use this code to show you how he
39:19
will use this code to show you how he
39:19
will use this code to show you how he will calculates at the end the total of
39:22
will calculates at the end the total of
39:22
will calculates at the end the total of our orders for example okay so I will
39:26
our orders for example okay so I will
39:26
our orders for example okay so I will rerun the last time this is the this is
39:28
rerun the last time this is the this is
39:28
rerun the last time this is the this is really the last time ah don't stop like
39:31
really the last time ah don't stop like
39:32
really the last time ah don't stop like that okay great so here we will see we
39:37
that okay great so here we will see we
39:37
that okay great so here we will see we will see how he will aggregate results
39:40
will see how he will aggregate results
39:40
will see how he will aggregate results and at the end he will print us the
39:43
and at the end he will print us the
39:43
and at the end he will print us the total of what we Ed okay so here I will
39:48
total of what we Ed okay so here I will
39:48
total of what we Ed okay so here I will try to call it very very very fast okay
39:52
try to call it very very very fast okay
39:52
try to call it very very very fast okay I have
39:54
I have
39:54
I have t-shirt let's do it
39:58
t-shirt let's do it
39:58
t-shirt let's do it let's do
40:00
let's do
40:00
let's do it
40:02
it
40:02
it great you I have one minute so clear it
40:07
great you I have one minute so clear it
40:07
great you I have one minute so clear it now I will call conference on t-shirt
40:10
now I will call conference on t-shirt
40:10
now I will call conference on t-shirt they have four I call pants with five
40:15
they have four I call pants with five
40:15
they have four I call pants with five and here after the 10 minutes will
40:18
and here after the 10 minutes will
40:18
and here after the 10 minutes will finish now he will make the calculation
40:20
finish now he will make the calculation
40:20
finish now he will make the calculation total nine so we aggregated and do some
40:24
total nine so we aggregated and do some
40:24
total nine so we aggregated and do some work to get us for example now 9 orders
40:28
work to get us for example now 9 orders
40:28
work to get us for example now 9 orders for the to for for the 10 seconds that
40:31
for the to for for the 10 seconds that
40:31
for the to for for the 10 seconds that we that we configured in our time based
40:34
we that we configured in our time based
40:34
we that we configured in our time based aggregator so thank you so much and see
40:40
aggregator so thank you so much and see
40:40
aggregator so thank you so much and see you bye-bye
40:42
you bye-bye
40:42
you bye-bye [Music]