Implement Distributed Rate Limiting in .NET 8 Microservices with Azure API Management and Redis Token Bucket

Why Distributed Rate Limiting Matters

Imagine a retail API that processes 12,000 requests per second during a flash sale. One overloaded instance can cause latency spikes, time‑outs, and even cascade failures across the entire service mesh. Distributed rate limiting guarantees that every microservice respects a global request quota, no matter how many pods or containers run behind a load balancer. In .NET 8 the built‑in RateLimiting middleware makes it easy to plug a custom algorithm, but the state must be shared – that’s where Redis and Azure API Management (APIM) become essential.

Token Bucket Algorithm Explained

The token bucket algorithm mimics a leaky bucket that refills at a constant rate. Each incoming request consumes one token; if the bucket is empty, the request is rejected or throttled. Compared with a fixed window counter, the token bucket smooths traffic bursts while preserving a strict average rate. For example, a bucket capacity of 100 tokens refilled at 20 tokens/second allows a client to send a rapid burst of 100 calls, then automatically throttles to 20 calls per second thereafter.

Preparing Redis for Shared Token State

Redis provides sub‑millisecond latency and native support for atomic operations, making it ideal for a distributed token store. Deploy a Redis Cluster in Azure (Standard tier, at least three shards) to avoid a single point of failure. Create a dedicated key namespace, e.g., rl:{clientId}, and store the current token count together with the last refill timestamp. Using the STRING type with a simple GET/SET pattern is sufficient, but Lua scripts can guarantee atomicity if you need to handle high concurrency.

Azure API Management Policy Integration

APIM sits at the edge of your microservice landscape and can enforce a first‑line rate limit before traffic even reaches Redis. Define a <rate-limit-by-key> policy that references a product or subscription key, then forward the request to the backend where the .NET 8 service applies the token‑bucket logic. This two‑layer approach protects against accidental denial‑of‑service attacks while still giving each service the flexibility to adjust limits per client.

.NET 8 Middleware Implementation

Below is a minimal implementation of a Redis‑backed token bucket as a custom IRateLimiterPolicy. The middleware reads the client identifier from a header (e.g., X‑Api‑Key), fetches the bucket, refills tokens based on elapsed time, and either grants a RateLimitLease or returns HTTP 429. Register the policy in Program.cs and chain it after authentication.

using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Caching.Distributed;
using System.Threading.RateLimiting;

public class RedisTokenBucketRateLimiter : IRateLimiterPolicy
{
    private readonly IDistributedCache _cache;
    private readonly int _capacity;
    private readonly double _refillRate; // tokens per second

    public RedisTokenBucketRateLimiter(IDistributedCache cache, int capacity, double refillRate)
    {
        _cache = cache;
        _capacity = capacity;
        _refillRate = refillRate;
    }

    public async ValueTask AcquireAsync(HttpContext context, CancellationToken token = default)
    {
        var clientId = context.Request.Headers["X-Api-Key"].ToString();
        var key = $"rl:{clientId}";
        var bucket = await _cache.GetStringAsync(key) ?? "0|" + DateTimeOffset.UtcNow.ToUnixTimeSeconds();
        var parts = bucket.Split('|');
        var tokens = double.Parse(parts[0]);
        var lastRefill = long.Parse(parts[1]);

        var now = DateTimeOffset.UtcNow.ToUnixTimeSeconds();
        var elapsed = now - lastRefill;
        tokens = Math.Min(_capacity, tokens + elapsed * _refillRate);
        if (tokens < 1)
        {
            return null; // limit exceeded, APIM will translate to 429
        }
        tokens -= 1;
        var newValue = $"{tokens}|{now}";
        await _cache.SetStringAsync(key, newValue, new DistributedCacheEntryOptions
        {
            AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(5)
        });
        return new RateLimitLease(); // simplified success lease
    }
}

In Program.cs add:

builder.Services.AddStackExchangeRedisCache(options =>
{
    options.Configuration = "myredis.cache.windows.net:6380,password=******,ssl=True,abortConnect=False";
    options.InstanceName = "RateLimiter";
});

builder.Services.AddRateLimiter(options =>
{
    options.AddPolicy<RedisTokenBucketRateLimiter>("RedisBucket", _ => new RedisTokenBucketRateLimiter(
        builder.Services.BuildServiceProvider().GetRequiredService<IDistributedCache>(),
        capacity: 100,
        refillRate: 20));
});

var app = builder.Build();
app.UseRateLimiter();
app.MapControllers();
app.Run();

Now every request passes through the token bucket, and the bucket state lives in Redis, making the limit truly distributed across all service instances.

Observability and Auto‑Scaling

Expose the current token count and refill timestamps through a health endpoint (e.g., /metrics) so Grafana or Azure Monitor can chart usage. Set up an alert when the 429 rate exceeds 2 % of total traffic – a signal that you may need to increase capacity or adjust the refill rate. Because the bucket capacity and refill are configurable at runtime, you can implement a feature flag that raises limits during a known promotion period without redeploying.

Conclusion

Combining Azure API Management’s edge policies with a Redis‑backed token bucket gives .NET 8 microservices a robust, low‑latency distributed rate‑limiting solution. The approach scales horizontally, survives node failures, and lets you fine‑tune limits per client or per product. By wiring the custom IRateLimiterPolicy into the built‑in middleware, you keep the implementation clean, testable, and fully integrated with the ASP.NET Core pipeline.

Sources

Microsoft Docs – ASP.NET Core Rate Limiting Middleware; Azure Documentation – API Management Policies; Redis.io – Data Types and Lua Scripting.

Author: Mahmut Sarıkaya — sarikayadev.com