Implement Distributed Tracing and Metrics in .NET 8 Microservices with OpenTelemetry, Azure Monitor, and Grafana

Why Distributed Tracing Matters in .NET 8 Microservices

Imagine a user request that hops from an API gateway to three downstream services before a response is returned. Without visibility, a single 500 ms latency spike can feel like a black box. According to the 2023 Cloud Native Computing Survey, 68% of engineers cite lack of end‑to‑end observability as the top barrier to scaling microservices. Distributed tracing fills that gap by stitching together spans from each service, while metrics reveal trends over time. In .NET 8, the built‑in minimal hosting model makes it easy to embed telemetry directly into the pipeline.

Combining OpenTelemetry with Azure Monitor and Grafana gives a complete feedback loop: OpenTelemetry collects data, Azure Monitor provides secure storage and alerting, and Grafana turns raw numbers into actionable dashboards.

Setting Up OpenTelemetry in a .NET 8 Service

The first step is to add the required NuGet packages. Use the .NET CLI on a Windows, Linux, or macOS development machine that meets the .NET 8 runtime requirement (minimum 2 GB RAM, 4 CPU cores).

dotnet add package OpenTelemetry.Exporter.AzureMonitor
dotnet add package OpenTelemetry.Extensions.Hosting
dotnet add package OpenTelemetry.Instrumentation.AspNetCore
dotnet add package OpenTelemetry.Instrumentation.Http
dotnet add package OpenTelemetry.Exporter.Prometheus

Next, configure OpenTelemetry in Program.cs. The example below registers both tracing and metrics, adds ASP.NET Core and HttpClient instrumentation, and ties the service name to Azure Monitor.

using OpenTelemetry.Trace;
using OpenTelemetry.Metrics;
using OpenTelemetry.Resources;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddOpenTelemetry()
    .ConfigureResource(resource => resource.AddService("OrderService"))
    .WithTracing(tracing => tracing
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddAzureMonitorTraceExporter(o => {
            o.ConnectionString = builder.Configuration["AzureMonitor:ConnectionString"];
        })
    )
    .WithMetrics(metrics => metrics
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddAzureMonitorMetricExporter(o => {
            o.ConnectionString = builder.Configuration["AzureMonitor:ConnectionString"];
        })
        .AddPrometheusExporter()
    );

builder.Services.AddControllers();

var app = builder.Build();

// Expose Prometheus endpoint at /metrics
app.MapPrometheusScrapeEndpoint();

app.MapControllers();

app.Run();

Notice the use of ConfigureResource to set a consistent service.name. This identifier appears in every span and metric, making cross‑service correlation trivial.

Exporting Traces and Metrics to Azure Monitor

Azure Monitor expects a connection string that includes the instrumentation key and the ingestion endpoint. Store this secret in appsettings.json or Azure Key Vault, never hard‑code it.

{
  "AzureMonitor": {
    "ConnectionString": "InstrumentationKey=YOUR_KEY;IngestionEndpoint=https://YOUR_REGION.monitoring.azure.com/"
  }

When the application starts, the OpenTelemetry exporters push data over HTTPS. Azure Monitor automatically creates a Log Analytics workspace where traces appear under the “traces” table and metrics under “metrics”. You can set alert rules—e.g., fire an email when the 99th‑percentile latency of the OrderService exceeds 1 second for five consecutive minutes.

Visualizing Data with Grafana

Grafana connects to Azure Monitor via the built‑in data source plugin. After adding the data source, create a dashboard that mixes trace latency heatmaps with Prometheus‑derived request rates. The following query shows the average request duration for the last 10 minutes:

avg_over_time(rate(http_server_requests_duration_seconds_sum{service="OrderService"}[10m]))

Combine it with a trace panel that visualizes the most recent spans tagged with http.method="POST". Grafana’s “Explore” view lets you drill down from a high‑level metric spike to the exact span that caused the delay, speeding up root‑cause analysis.

Best Practices and Common Pitfalls

Sample rate management: Capturing every request can overwhelm both Azure Monitor and Grafana. Set OTEL_TRACES_SAMPLER to parentbased_traceidratio with a 0.2 (20%) ratio for production workloads.

Context propagation: Ensure that outbound HttpClient calls carry the trace context. The OpenTelemetry HttpClient instrumentation does this automatically, but custom HttpMessageHandlers must call Activity.Current?.SetParentId(...) if they create new activities.

Version alignment: Use OpenTelemetry packages that target .NET 8 (v1.5.0 or later). Mixing older versions can cause missing attributes or runtime errors.

Conclusion

Implementing distributed tracing and metrics in .NET 8 microservices no longer requires a patchwork of third‑party SDKs. By standardizing on OpenTelemetry, exporting to Azure Monitor, and visualizing with Grafana, teams gain a unified observability stack that scales with the cloud. The key steps are: add the right packages, configure resource attributes, export securely, and tune sampling. With those foundations in place, performance problems become visible before they affect users.

Sources

Microsoft Docs – OpenTelemetry for .NET
Azure Monitor documentation – Exporters
Grafana Labs – Azure Monitor data source guide

Author: Mahmut Sarıkaya — sarikayadev.com