The Durable Task Framework (DTFx) allows users to write long running, persistent workflows in C# using the .Net framework and simple async/await coding constructs.
These workflows are also called orchestrations and require at least one provider (a backend persistence store ) for storing the orchestration message and runtime state. It provides a low-cost alternative for building robust workflow systems. The open-sourced Microsoft library is accessible here.
In this post, let’s understand how the Durable Task Framework works.
Highlights:
|
What Problem Can a Durable Task Framework Solve?
The framework is designed to allow users to build workflows using code. If processing can take a long time or one of the business requirements is to prepare a state machine or use a workflow engine, the Durable Task is an ideal option. It can also be a link between many microservices or act as a distributed component.
Key Features Of The Framework
- The solution is lightweight and simple.
- Workflows are defined in code, which makes them customizable.
- The DTFx uses the Event Sourcing approach to store all actions in Azure Table Storage. Thanks to this feature, we can rerun the processes from the past or re-initiate the process from a state where the activity was interrupted.
- The persistent state feature of DTFx allows for managing long running complex processes and helps to reduce the time duration.
- Versioning of orchestrations and activities helps track changes.
- DTFx supports an extensible set of backend persistence stores such as Service Bus, Azure Storage, SQL Service, Service Fabric, etc., and requires minimal knowledge of the services.
Why Is It Important In Cloud-Native Application Development?
Cloud Native Applications promote a loosely coupled, microservices-based application architecture. The Durable Task Framework enables the building of such applications, where most of the work in building a complex workflow system is done by the DTFx. Today, all leading cloud solution providers offer a Durable Task Framework, making it a cost-effective alternative for building high-performance workflow systems.
- Durable Functions and Azure Functions are good examples of how the Durable Task Framework is being extended to offer cloud-native solutions on Azure.
- AWS SWF (Simple Workflow Framework), a web service that facilitates coordination of work across distributed application components, is another good example of using the Durable Task Framework to coordinate tasks, manage execution dependencies, schedule and handle concurrency in accordance with the application’s logical flow
- Google Cloud Tasks is a fully managed service that lets you manage the execution, dispatch, and delivery of a large number of distributed tasks.
How Does Azure Durable Task Work?
The Durable Task Framework consists of several components that work together to manage and execute orchestrations and activities. Let us take a look at the different components and how this is accomplished.
Task Hub: It is a logical container for Service Bus entities used by the Task Hub Worker to reliably pass messages between code orchestrations and the activities they orchestrate.
Task Activities: These are pieces of code that perform specific steps of the orchestration. A Task Activity can be ‘scheduled’ from within some Task Orchestration code. This scheduling results in a plain vanilla .NET Task that can be serviced (asynchronously) and that can be composed with other similar Tasks to build complex orchestrations.
Task Orchestration: This is where you can schedule Task Activities and build code orchestrations around the Tasks that represent the activities.
Task Hub Worker: This hosts Task Orchestrations and Activities. It also contains APIs for performing CRUD operations on the Task Hub itself.
Task Hub Client
The Task Hub Client provides:
- APIs to create and manage task orchestration instances
- APIs to query the state of Task Orchestration instances from an Azure Table
The Task Hub Worker and Task Hub Client are connected to the Service Bus and Azure Table Storage via connection strings. The Service Bus is used for storing the execution control flow state and passing messages between Task Orchestration instances and Task activities.
Since the Service Bus is not meant to be a database, the state is removed from the Service Bus once the code orchestration is complete. However, if an Azure Table storage account is linked, this state is available for queries as long as the user stores it.
The framework provides Task Orchestration and Task Activity base classes from which we can derive to specify orchestrations and activities. Then use the Task Hub APIs to load these orchestrations and activities into the process and then start the worker that begins processing requests to create new orchestration instances.
The Task Hub Client APIs are used to create new orchestration instances, query existing instances, and terminate those instances as needed.
We start by creating a new orchestration instance that loads all activities into the Service Bus as a control flow, based on the orchestration definition. All activities are then executed one by one. This process ensures that all actions are invoked only once. If something goes wrong with the application and the problem is resolved, the framework resumes from the exact activity it was executing at the time of the crash.
Basic Sample Of Durable Task Framework Implementation
This solution aims to demonstrate DTFx using a simple scenario of a Customer Sign up. The Task Orchestration is the complete signup process and the Task Activities are different steps performed to achieve this.
We have 4 Task Activities: Address Check (PerformAddressCheckAsync), Credit Check (PerformCreditCheckAsync), Bank Account Check (PerformBankAccountCheckAsync), and Sign Up Customer.
The three checks are executed in parallel, and once they are completed, the Signup Customer Task is invoked. This returns the ID only if all checks were successful, or rejects it with a ‘REJECTED’ message if one or more checks fail.
The Credit Check Task is a data-driven asynchronous process. It relies on the ‘NumberOfCreditAgencies’ value passed as input to the workflow.
In Figure 1, we see that 3 activities are Async Methods that return the Task result of type Boolean. Each of these methods calls the corresponding sub-methods in Figure 2. This performs the validation of the Address, Bank Account, and Check Credit Score. In addition, we have the 4th activity, Signup Customer, which generates and returns the Customer ID.
Fig. 1
Fig. 2
Figure 3 shows the implementation of Task Orchestration. Thus, each created orchestration instance executes all declared activities. Based on the result of the checks, the last activity returns the result as ‘User Signed up ID’ or ‘Rejected’.
Fig. 3
Key Takeaways
- With the Durable Task Framework, Microsoft provides a simple option for building robust, distributed, and scalable services. DTFx has built-in state persistence and program execution checkpoints.
- By implementing The Durable Task Framework, you can manage all system logic from one place and define the process flow.
- The Durable Task Framework uses Event Sourcing to store all actions in Azure Table Storage. You can resume the processes from the past or continue the process from the point where the action was interrupted.