Build a Serverless Event‑Driven Workflow Engine with .NET 8, Azure Durable Functions, and Service Bus

Why event‑driven serverless workflows are gaining traction

Enterprises that processed more than 10 million messages per month in 2023 reported a 30% reduction in operational overhead after moving to an event‑driven, serverless model. The combination of .NET 8, Azure Durable Functions, and Azure Service Bus gives developers a predictable cost curve while preserving the agility of micro‑services.

Core concepts of an Azure Durable Functions workflow

Durable Functions extend the classic Function‑as‑a‑Service model with stateful orchestrations. An orchestrator function defines the workflow graph, while activity functions perform the actual work. The runtime stores state in Azure Storage, enabling automatic replay and exactly‑once execution without any developer‑managed database.

Key benefits for a .NET 8 codebase include native async/await support, source‑generator‑based function signatures, and seamless integration with the latest System.Text.Json serializer.

Designing the event‑driven architecture

In a typical order‑processing scenario, three events flow through Service Bus topics: OrderCreated, InventoryReserved, and PaymentCharged. Each event triggers a separate activity, but the orchestrator guarantees the correct sequence and compensating actions if a step fails.

Because Service Bus guarantees at‑least‑once delivery, the orchestrator must be idempotent. Using the durable context’s GetInput<T>() method you can pull the correlation ID from the incoming message and store it in the orchestration state.

Setting up the development environment

System requirements: Windows 11 or Ubuntu 22.04, .NET 8 SDK, Azure CLI 2.55+, and an Azure subscription with a Resource Group.

Installation steps (run in a terminal):

dotnet new console -n ServerlessWorkflow && cd ServerlessWorkflow dotnet add package Microsoft.Azure.Functions.Worker.Sdk dotnet add package Microsoft.Azure.Functions.Worker.Extensions.DurableTask dotnet add package Microsoft.Azure.WebJobs.Extensions.ServiceBus

After restoring packages, create a local.settings.json file with the Service Bus connection string and the storage account connection string required by Durable Functions.

Implementing the orchestrator and activities in .NET 8

The orchestrator receives the order identifier from a Service Bus trigger, then calls three activities in sequence. The code below demonstrates the latest C# 12 features, such as file‑scoped namespaces and global using directives.

using System.Threading.Tasks; using Microsoft.Azure.WebJobs; using Microsoft.Azure.WebJobs.Extensions.DurableTask; using Microsoft.Azure.WebJobs.Extensions.ServiceBus; using Microsoft.Extensions.Logging; namespace ServerlessWorkflow; public static class OrderWorkflow { [FunctionName("OrderOrchestrator")] public static async Task<string> RunOrchestrator( [OrchestrationTrigger] IDurableOrchestrationContext ctx, ILogger log ) { var orderId = ctx.GetInput<string>(); await ctx.CallActivityAsync<string>( "ValidateOrder", orderId ); await ctx.CallActivityAsync<string>( "ReserveInventory", orderId ); await ctx.CallActivityAsync<string>( "ChargePayment", orderId ); return "Completed"; } [FunctionName("ValidateOrder")] public static async Task<string> ValidateOrder( [ActivityTrigger] string orderId, ILogger log ) { // Simulate validation log.LogInformation($"Validating order {orderId}"); await Task.Delay(500); return "Validated"; } [FunctionName("ReserveInventory")] public static async Task<string> ReserveInventory( [ActivityTrigger] string orderId, ILogger log ) { log.LogInformation($"Reserving inventory for {orderId}"); await Task.Delay(700); return "Reserved"; } [FunctionName("ChargePayment")] public static async Task<string> ChargePayment( [ActivityTrigger] string orderId, ILogger log ) { log.LogInformation($"Charging payment for {orderId}"); await Task.Delay(600); return "Charged"; } [FunctionName("OrderCreatedTrigger")] public static async Task TriggerOrchestrator( [ServiceBusTrigger("order-created", "order-subscription", Connection = "ServiceBusConnection")] string message, [DurableClient] IDurableOrchestrationClient client, ILogger log ) { var orderId = message; var instanceId = await client.StartNewAsync("OrderOrchestrator", orderId); log.LogInformation($"Started orchestration with ID = {instanceId} for order {orderId}"); } }

Notice the use of StartNewAsync inside the Service Bus trigger, which decouples the ingestion point from the orchestration logic.

Connecting Durable Functions to Azure Service Bus

Service Bus topics act as the event backbone. Define three subscriptions—one per activity—if you need fine‑grained scaling. In most cases a single subscription feeding the orchestrator is sufficient, because the orchestrator itself dispatches the downstream calls.

When configuring the function app, set functionsExtensionVersion to ~4 and enable the extensionBundle for Durable Functions. This ensures that the runtime automatically provisions the required storage accounts.

Testing and monitoring in production

Local testing can be performed with the Azure Storage emulator and the Service Bus “az servicebus namespace authorization-rule keys list” command to fetch a shared access key. Use the Azure Functions Core Tools func start command; the orchestrator logs will show the replay behavior whenever a new event arrives.

In production, Azure Monitor and Application Insights provide end‑to‑end tracing. The built‑in “DurableTask” telemetry surface includes orchestration duration, activity latency, and failure rates. Set an alert on the “DurableTask Failed” metric to catch compensating‑action loops early.

Best practices and pitfalls to avoid

1. Keep activity functions short (under 5 seconds) to stay within the default consumption plan limits. 2. Use idempotent designs: store a processed flag in Azure Table Storage or Cosmos DB keyed by the correlation ID. 3. Avoid long‑running loops inside an orchestrator; instead, schedule a timer with ctx.CreateTimer and let the function return.

4. When scaling out, set the Service Bus MaxConcurrentCalls property to match the expected throughput—e.g., 200 for 10 k messages per minute. 5. Regularly purge completed orchestration instances older than 30 days to keep the underlying storage tidy.

Conclusion

By leveraging .NET 8’s performance improvements, Azure Durable Functions’ stateful orchestration, and Azure Service Bus’s reliable messaging, developers can build a robust, serverless, event‑driven workflow engine in less than a week. The pattern scales automatically, reduces operational cost, and keeps business logic cleanly separated from infrastructure concerns.

Sources

Microsoft Docs – Azure Durable Functions; Microsoft Docs – Azure Service Bus; Azure Architecture Center – Event‑driven serverless patterns

Author: Mahmut Sarıkaya — sarikayadev.com