Up next in 10
Azure Durable Functions - Stateful workflow on top of Stateless Architecture by Debasis Saha
Oct 30, 2023
C# Corner - Community of Software and Data Developers
https://www.c-sharpcorner.com
Conference Website: http://conference.c-sharpcorner.com
#CSharpCorner #CSharpConf22
About C# Corner
C# Corner, headquartered in Philadelphia, PA, is an online global community of 3 million software developers. C# Corner serves 5+ million visitors with 9 million page views each month. We publish the latest news and articles on cutting-edge software development topics. Developers share their knowledge and connect via content, forums, and chapters. Thousands of members benefit from our monthly events, webinars, and conferences. We also provide tools for career growth such as career advice, resume writing, training, certifications, books and whitepapers, and videos.
Show More Show Less View Video Transcript
0:13
Hi everybody. I am audible, right? Yeah. So today's session is about the Azure Durable Functions
0:22
So in this session we will discuss about the Azure Durable Functions
0:27
So, I am Devasi Shah, basically working as a solution architect and a CSup Corner MVP
0:34
along with Microsoft certified trainer. So, in this session, we are discussing on different topics like concept of Azure functions
0:43
concept of Azure Durable functions, what is the purpose of using Azure Durable functions
0:48
why we should use it, or what are the benefits of using, and if we are wanting to use the
0:53
the Azure Durable Functions, then what will be the structure or applications pattern we
0:58
need to follow. So those will be discussed about in the sessions, obviously in a very
1:03
short manner, because we have a limited timing. And also at the end we'll demonstrate how
1:10
we can create or develop Azure Functions. Okay, so before going to start about the Durable
1:17
functions or how it will work, we first need to understand about the Azure functions. So
1:25
when we talk about the Azure functions, first thing, if we search in the Google, so what
1:30
is Azure functions, the first keyword which is coming that Azure functions basically is
1:35
a serverless technology. So the concept is what is serverless. Serverless means earlier
1:41
days when we develop any web applications or any type of applications, we always need
1:46
to install or host that applications in a particular server. Either it's a cloud server
1:51
or it's an on-premise service, everything. But once the cloud environment appears, then
1:57
the concept of software development or the software development implementation has been
2:02
changed. Now, most of the companies don't want to use their own server or maintain their
2:07
own hardware system because as a company, if I want to use my own server, in that case
2:13
I need to maintain everything. It means hardware, maintenance teams, every headache is going to our company head
2:20
So most of the companies are now prefer to use the cloud environment because the infrastructure
2:26
headache or infrastructure related implementation does not require because the provider who
2:31
provides the cloud environment support, they actually take their responsibility on behalf
2:36
of the company. So in this point of view, the term serverless technique is appear. Serverless
2:42
techniques as a developer, it is for me as a serverless technique, but as a whole, it's
2:49
basically there is nothing called serverless technique. Because without server, we cannot
2:53
use any software. Whatever software we are using or any web applications or anything
2:59
It must be installed or hosted in any particular server. But as a developer, why is it serverless
3:05
Because once I develop the applications, I do not wait to host it in any applications
3:11
or in a server. I simply host that applications in the cloud environment or deploy the cloud environment
3:18
and directly use it. So I do not ask to my senior teams or management team that, okay, I have completed my development
3:24
now provide me the server access so that I can install the software in the server or
3:29
I can host the applications. So that from this term or concept, the serverless techniques
3:34
is appeared. So serverless is basically is it totally depends upon the cloud computing
3:40
executions model. Okay. But ultimately what I said that it is basically not a serverless
3:46
it basically has a server, but that server is basically maintained by the cloud provider
3:50
by us. So who are the consumer? It means us, companies, user, developer who are the consumer
3:56
they do not maintain the server. The provider or the cloud provider basically maintains
4:02
the server. So in case of serverless techniques, we have two different approaches which we
4:09
can follow. One is the function as a service or fast model or as a microservice. These
4:16
are the two different approaches we can use in case of serverless compute system in a
4:21
cloud environment. So in that case, majorly is that we can just simply need to develop
4:27
our business applications as a function or as a block of code. We do not even need to
4:33
implement the entire applications or entire boilerplate architecture of a particular application
4:38
in case of function as a service. We just need to implement the exact requirement of
4:44
our implementations. It means if I have a requirement that this particular task needs
4:49
to be done, then as per that task or as per that requirement, I need to implement the business
4:54
logic, write down the code, and then host that code as a function in the cloud. So that's
5:00
why it is called as a function as a service. And the cloud provider, rest of the other
5:06
part is provided by the cloud provider. So when that particular function is used by access
5:14
the users. In that time, if the server is required to scale up or scale down, like say
5:21
maybe at the initial I have assumed that per minute or per second, 100 users will consume
5:26
my service. Okay. But certain scenario that number of users increase to 1 lakh. So in
5:32
that case, the cloud provider automatically scale up or down. So that depends upon the
5:36
usability of the service or the function. So as a developer or as a consumer, we just
5:44
Don't worry about this. So in this concept, there are two different architecture we can follow
5:49
One is the Azure logic apps and another is the Azure functions. So today's session is basically depends upon the Azure functions
5:58
So what is the Azure function? Azure functions is basically represents a block of code as a serverless architectural
6:05
pattern where we just need to host our business logic which can be executed without any provided
6:13
infrastructure. Suppose if we want to implement, I'll give you one practical example, suppose
6:19
we are wanting to implement some approval process for our own company. Like as for example
6:25
the leave management system. So we want that some employee apply for a leave, that one
6:31
notification will need to go to the reporting manager immediately and after certain durations
6:37
of time, say within three days or after seven days, if the reporting manager is not approved
6:42
that applications or does not take any initiative against that applications, then maybe reminder
6:47
needs to be sent or that needs to be accelerated in the next level of employee
6:53
So if we want to develop these types of mechanism or workflow in that case if normal software part or as a web application we need to implement everything We need to capture the applications everything we need to capture
7:06
But in that case, in this case we just need to implement the application logic or that particular workflow which we need to be executed
7:14
So that workflow automatically is initiated when the particular application has been made
7:20
So whenever the particular employee initialized or applied for a leave or applied for the
7:26
leave request, then automatically that workflow initialized. And then it automatically continued as part of the workflow logic
7:33
After seven days, it automatically checks the state of that application that either it
7:38
is closed or approved or declined, anything. If it is not, then as part of the workflow definitions, it automatically executes the
7:46
next level. Either there is a reminder or say there is an exclamation mail, anything, whatever we
7:53
have defined in the workflow, that will be automatically executed. And this entire state does not need to maintain by our applications
8:00
It automatically done by the durable functions or Azure functions. So these functions provide instant scalability, which I have already said, and most importantly
8:11
this will be charged only for the resource use. is suppose I have implemented functions and hosting the Azure platform. As we know, in
8:22
Azure cloud, everything is pay by use. Whatever resources we have used, that we need to pay
8:29
But suppose I have deployed functions in the cloud, but next seven days, none of the users
8:36
use their functions. Means there is no consumption. Maybe this is a legal application approval
8:42
process. So next seven days there is no leave application has been done. Or there is nothing
8:46
pending from the existing system. So in that case, none of the user has consumed or none
8:53
of the workflow is executed. That means as a company, I does not pay for that during
8:58
some period. Whenever any workflow is initiated or any user or system is executed, you can
9:04
say some data or some user is consumed at function, only during that part or during that timeframe
9:11
we need to pay or Azure will be charged for us. So, and one of the major benefits of Azure
9:19
function is it basically provides different language. We can develop Azure functions by
9:23
using different languages like C-SERP, we can use EPSERP, JavaScript, Python, and also
9:29
the PowerShell. So, there are different languages we use. So, for developing Azure functions
9:35
It is another benefit that we don't require to learn new techniques or new programming language
9:43
If I am a C-Sub developer, I can develop the language by using the C-Sub. If I am a JavaScript or say React developer or jQuery developer, I can develop in my
9:52
own language. Similarly, for Python, if I am a data engineer and I am working on a Python language, then
9:57
I can develop it also in the Python. Similarly PowerShell and F-Sub is also supported
10:02
So right now, Azure functions provided these five languages, but hopefully in the near
10:06
future, it will introduce more programming languages. And also, if during the execution of functions, if we require any third-party package manager
10:16
like NuGet or NPM, NuGet obviously we know the CSAB or Java, FSAB developer, we know
10:22
the NuGet packages. And for JavaScript or React, NPM is one of the best, which provides different types of
10:29
third-party or open-source projects or tools. So we can reuse them here. So suppose any functionality we require and we have a NPM
10:38
package, we know that this functionality is available in that particular NPM package and
10:42
if you use that, we don't need to write down everything. So we can directly use it and
10:47
we can, while installing the functions in the cloud, the same package manager will be
10:52
automatically installed. That's what we need to mention in the configuration layer. And
10:57
basically based on the trigger and binding. Trigger and binding means, trigger means whenever any user invoke that function
11:04
Function is basically just like a gun. So if I have a gun, it will be harmless
11:14
whenever you not trigger the button. So whenever anyone is hit the functions
11:19
or call the functions, then it trigger and automatically start executing. And binding means the data
11:25
If I send some data to functions, then it will bind and implement the states
11:30
If suppose there is a single purpose objective, means it will execute, check some status or data
11:36
and return me the result. So then binding is not required. Binding when required, when I need to maintain the state
11:42
State means, suppose one app, just I have given the example, so leave application has been done
11:48
but after three days or seven days, I need to check that, what is the status of leave application
11:53
So this is called a binding. So whenever application is done, it will trigger, and then it will continue until that binding
12:00
or that trigger has not end. So whenever that particular application is active, it automatically continues binding
12:08
That is called a state. Because in normal concept, Azure function is basically stateless
12:12
There is no memory management system in Azure functions. It basically is a single point execution
12:18
It executes, finds some data as per the business logic, calculate, and return you the output
12:24
This is the simple logic of Azure Functions, one-liner. But in case of durable functions, it always maintain a state because durable functions
12:31
is basically not a single Azure Functions, it is represented by a single one functions
12:36
Whereas durable functions is basically, so it is basically the extension part of the
12:45
Azure Functions. So durable functions is basically long-lasting, stateful applications. So in case of Azure function, it's basically a single execution point. And whenever we
13:00
install or upload any Azure function, it's always a single function. It means if we are
13:06
a C sub, whatever, any type of functions we can develop. So if I have defined two different
13:12
functions, that means in Azure cloud, it will create two different functions. So in each
13:17
provided by a different URL. But in case of durable functions, it basically processes
13:25
this. So it basically taken data from the input parameters or from the database, whatever
13:31
we have mechanism, process that data and perform the appropriate actions. So this is basically
13:37
one-liner definitions of Azure functions or responsibility of Azure functions. But in
13:42
In case of durable functions, it basically is not a single function
13:46
It basically is a series of functions where we can define a combinational chain, different
13:51
types of series. That say I can define five different functions and then we mention that okay first execution will be the function 1 if function 1 result is this Suppose if function is the one return me a boolean value So result idea true or false so if return is true then function 2 will execute it
14:09
But if result is false then function 3 will execute it So this type of chain executions can be done in case of durable functions only
14:16
So if we are doing only the Azure functions and it's a single function means single function in single tags
14:22
I will take some inputs, process it and give you the result
14:26
So on that result, taking that result, what will be happening? That will not be responsible for Azure functions
14:32
But in case of durable functions, durable functions maintain the entire response with that
14:37
Okay, user gives the input and after processing the different sets of results, as per the
14:42
result in the workflow, we define that, okay, if result is like this, then this step need
14:48
to be done. not this, then this needs to be, then accordingly to act accordingly. So that's why it maintains
14:53
the intercept. And this function or execution channel can be maintained as many as we can
15:01
mention in the workflow. So that's why we always mention that durable functions always
15:06
maintain a workflow process. So durable functions programming model is very simple and flexible
15:14
like Azure functions there is nothing different. When we write down the, create the durable
15:20
functions we can perform the rapid programming language, rapid process like we just want
15:26
to maintain the, eliminate the boilerplate or we just want to maintain the entire application
15:31
structure means I just want to initialize the program, initialize the, then loading
15:36
the program, everything I just want to mean. I just need to write down the main workflow
15:41
And also, we can create the part by using the portal. Yeah, definitely we can develop the Azure Durable Function
15:49
by using different types of ID, like Visual Studio, Visual Studio Code, Eclipse
15:55
There are different IDs supported by Azure Functions or Durable Functions. But if as a developer does not want to use them
16:02
I simply develop the Azure Functions or Durable Functions within the Azure Portal
16:08
because Azure Portal, it provides me a text editor where I can write down my code and execute it
16:13
in the DirectLine portal. So for developing the Azure functions or durable functions
16:17
we does not need any software to install in our computer. We can simply develop it online system
16:23
Also domain cost effective pricing that it pay for what we can use
16:28
And there is obviously no server concept because everything is hosted in Azure
16:33
so Azure will maintain the pattern. And also we can use different flavor of Azure web apps
16:41
like CI-CD, easy authentication, certificates, custom domain settings. These are the different functionalities provided by the Azure web applications
16:50
Azure web app means if we have developed any web applications and they're hosted in the Azure
16:55
those are normally treated as the Azure web apps. So whatever the settings or functionalities we are implementing there
17:02
the same thing we can also host it or use in the Azure Durable Functions applications also
17:07
So the main question is why should we use? Because we have the web applications, we have the Azure function and everything
17:15
But still why we need to use the durable function? The main question is it's basically event-driven
17:21
Obviously, the event-driven mechanism is also a feature of the Azure function, same way
17:27
So if we are going to use the Azure functions, then the same reason we are using the Azure
17:33
of the durable function, it is even driven. But the second one is not related to the Azure function
17:38
That means we can define our own workflow, means work process change, our own
17:43
as per required our own process change, we can define in case of durable functions
17:49
which we cannot define in case of normal Azure functions, because normal Azure functions is single tax executed
17:56
whereas Azure durable function is a chain process. There are different types of chain systems
18:00
or workflow you can define as per our requirement. So Microsoft or Azure does not provide any specific template or mechanism for defining
18:09
a workflow. That workflow must be defined by the user or developer as per their custom requirement
18:14
Yeah, obviously Azure provides different types of workflow process. Anyone we can use
18:21
But definitions of their workflow should be maintained by us, not by the cloud
18:28
And workflow management state. So when we define the workflow management or workflow process, we can maintain our own
18:37
managed workflow state. Means the durations or the value, how many days it needs to be deleted like this
18:44
Some of the companies, we have some mechanism that if suppose we have some mail and that
18:49
mail needs to be deleted after 30 days. Or we are not making the database overload, so after 30 days or after six months, the data
18:58
needs to be archived in another part. So that can be also a part of the as a durable functions
19:03
And this is the basically state. That means one record, one's inserted in the database
19:07
that state is maintained until six months. Because once that state is reached
19:12
then that next activity will be executed. So during this period, that state is checking every day
19:18
that either that particular record is reached the state conditions or not
19:22
So this workflow state can be maintained by us. There is no hardcore rules that, okay
19:28
we need to maintain this within 5 days. No. And testing and debugging. This is one of the
19:35
major benefits because as we can use different languages like C-Sharp, F-Sharp, and if we
19:42
are going to use our own ID, where we are familiar, like if I am a C-Sharp developer
19:47
obviously I prefer to work on Visual Studio or Visual Studio Code. If I am a Python developer
19:52
to work in either Visual Studio Code or in Eclipse. And because using those IDs, I can debug my program
20:01
without before hosting the applications. The same thing also can be done in case
20:05
of Azure Durable Functions also. As those languages supported by the existing ID
20:10
so we can develop our Azure functions in our ID by using the Azure ID, and we can debug it
20:17
before hosting that so that we can 100% see that logic or workflow which I define, that is work properly. So as I already said, these
20:28
are different environment or ID we can use Visual Studio, Azure portal, Visual Studio
20:33
code, interlink ID, Eclipse everywhere, we can use the Azure Durable functions. For Eclipse
20:41
or Visual Studio code, we need to install extensions related to this, but for Visual Studio, it
20:47
by default. Whenever we install the Visual Studio, by default we can have a level this
20:52
template Now when we developed Azure Durable Functions Azure Durable Functions basically contains three different types of functions because Azure Durable Functions is a process or workflow
21:05
so we need to define which type of functions are there. So Azure Durable Functions always contain
21:10
three different types of functions. First one is the client functions. Okay, as per our requirement, we can define
21:17
when we define any functions within the Durable Workflow state, we need to mention which type of function is this
21:23
This is not mandatory that we need to create every function type 1 is to 1
21:28
No, we can create the same type of function multiple times or multiple numbers
21:32
But that aspect of our requirement. Client function means whenever Azure Durable functions work, that means some particular
21:40
event will occur or particular activity has been happening. So that activity obviously happened in any other software, not my software, because I'm
21:50
not going to develop the entire software. This means, suppose if I want to implement a leave approval process, that does not mean
21:58
that I am going to create a leave management system where employee, I will define the leave
22:02
heads, leave balance, tag employee with the leave heads, and then employee can get a UI
22:09
where they can select a date from date to date, reasons everything, reporting authority
22:14
then apply. This entire process I am not going to use because this already, maybe, already have a software
22:20
which company already used because they have maybe different portal they are using. So
22:25
our requirement is whenever employee click on the apply button, Azure functions will
22:32
be invoked. That means I need a connector from that application to my Azure functions
22:38
which can establish this connection. So that connector will be provided by the client functions
22:43
So this type of third party invoker, whenever we require, then we need to define a client
22:49
functions. That client functions basically provide a URL. Each and every client functions
22:55
when we execute it in the portal or host in Azure, it basically provides us a URL. We
23:01
simply need to call that URL from our existing applications with provided payload, means
23:08
the existing data. Okay. Maybe sometimes we are not providing the payload because maybe
23:13
from where we need to provide the data, we know. So we are simply providing maybe the
23:18
ID or some reference number so that only that value, I'll pick up my own data here
23:23
So that is the client functions. Client functions are always executed by the third party, not by the entire process of the work
23:29
Whenever the client functions executed, the workflow starts. So it acts as the entry point
23:37
The next function is the orchestra functions. Whenever the client functions executed and then Azure functions workflow is started
23:45
That is workflow started mechanism or the duration of the workflow or the logic of workflow
23:49
means what are the process change of the workflow means whenever client function executed what
23:54
will happen after that execution how many days I need to wait or suppose what is the
24:00
result of this as part of the execution what will be done
24:03
So in that this workflow will be defined under orchestrator functions. So this is the internal part
24:09
This is not exposed to the outer world. function is the internal part and every Azure Durable Function workflow must have a orchestrator
24:17
function, one single orchestrator functions. This cannot be a multiple because this is
24:21
basically contained a workflow mechanism. So in a particular process change, workflow
24:27
mechanism cannot be multiple. You can define entire workflow within this function that
24:32
this is the logic. Once the data is executed, calculate this or check this. If result is
24:37
There is a, we can say there are five different combination sets of results company provided us
24:43
So according to that, I'll define five different sets of workflow. And for each and every workflow state, I'll create the next function, that is the activity function
24:55
So activity function means once the workflow is executed, then what activity will be happening
25:01
Means I have set that five different states. Okay. for each and every step I have a different activity
25:08
So step one, if the value is one, then I need to mail someone
25:13
If value is two, then I need to mail. So this activity can be executed
25:17
Okay, so that means in a durable functions workflow process, you have one single entry point
25:24
one single worker setter functions for the workflow, and a number of activity functions
25:30
You can define any number of activity functions. So there are different patterns like this
25:35
This is the one example. This is what we are calling the change system
25:39
That means function one is the entry point. Function one is executed
25:43
It executed and sent a result to the orchestra functions. And then as part of the result of function two
25:50
it will again execute the function three. It's a change system. So it's a one-liner system, one-by-one process step
25:56
Another process we call fan in or fan out. So, pan in pan out means one function is executed
26:04
we get some result, and after that, if we require to call three different functions
26:09
so F2 is, there's three different functions we are calling in a parallel thread
26:14
And after execution of three, all the functions, we'll ultimately summarize the final result
26:20
and that result will pass to the F3 for final decisions. So this is the one pattern
26:24
but not necessary we need to always follow this pattern. This is some example pattern, which we can follow
26:30
One other is the human interactions. That means request approval after some time interval
26:37
we'll check it that either request approval is processed or not. If the time interval within that part
26:43
request is not approved, then we need to escalate. And after escalate, if something is again need to define
26:48
then we can define again, same way. So these are the some different patterns we can use. Okay
26:57
So whenever, So whenever we host these things, automatically it will work
27:25
So for the example part, I have uploaded a sample example in this GitHub repository
27:31
If required, you can check this. And whatever the query regarding that code block
27:36
if you have any query or you need to discuss it further, then you can connect me in any of the mentioned addresses
27:45
Either you can mail me or pick in a LinkedIn or in Twitter directly
27:49
So we can discuss and explain. So that's all for these sessions
27:55
Thank you
#Networking
#Distributed & Cloud Computing
#Cloud Storage