Build RAG Pipelines with .NET 8, Azure OpenAI, and Pinecone

Why RAG is becoming the backbone of modern AI assistants

Did you know that 78% of enterprises plan to adopt Retrieval‑Augmented Generation (RAG) by the end of 2025? The promise of combining large language models with up‑to‑date, domain‑specific knowledge makes RAG the go‑to architecture for chatbots, knowledge bases, and decision‑support tools. Yet many .NET developers wonder how to stitch together Azure OpenAI, a vector database, and the latest .NET 8 features without writing a monolithic service.

Prerequisites and system requirements

Before writing a single line of C#, ensure you have the following:

• Windows 11 or a recent Linux distribution with .NET 8 SDK installed (download from dotnet.microsoft.com).
• An Azure subscription with access to Azure OpenAI Service (GPT‑4 or gpt‑35‑turbo).
• A Pinecone account (free tier is sufficient for prototyping) and an API key.
• Visual Studio 2022 17.9+ or VS Code with C# extensions.

All components run on x64 CPUs; no GPU is required for inference because Azure OpenAI handles the heavy lifting.

Setting up Azure OpenAI in .NET 8

The new Azure.AI.OpenAI NuGet package supports async streaming and typed request objects. Create a console project and add the package:

dotnet new console -n RagDemo && cd RagDemo && dotnet add package Azure.AI.OpenAI

Store your endpoint and key in user‑secrets to avoid hard‑coding credentials:

dotnet user-secrets init && dotnet user-secrets set "OpenAI:Endpoint" "https://YOUR_RESOURCE.openai.azure.com/" && dotnet user-secrets set "OpenAI:Key" "YOUR_API_KEY"

Inject the client via dependency injection in Program.cs:

using Azure.AI.OpenAI; using Microsoft.Extensions.Configuration; var builder = WebApplication.CreateBuilder(args); var config = builder.Configuration; builder.Services.AddSingleton(_ => new OpenAIClient(new Uri(config["OpenAI:Endpoint"]), new AzureKeyCredential(config["OpenAI:Key"]))); var app = builder.Build(); app.MapGet("/generate", async (OpenAIClient client, string prompt) => { var response = await client.GetCompletionsAsync("gpt-35-turbo", new CompletionsOptions { Prompt = { prompt }, MaxTokens = 200 }); return Results.Ok(response.Value.Choices[0].Text); }); app.Run();

This minimal API listens on localhost:5000 and returns a generated answer for any prompt.

Integrating Pinecone vector store

Pinecone provides a RESTful API that can be called from .NET using HttpClient. First, install the System.Text.Json package for serialization:

dotnet add package System.Text.Json

Define a simple model for upserting embeddings:

public record PineconeUpsertRequest(string[] ids, float[][] vectors, string[]? metadata = null); public record PineconeQueryResponse(string[] matches);

Assume you have an embedding function that converts text to a 1536‑dimensional vector (Azure OpenAI embeddings). The upsert method looks like this:

public async Task UpsertAsync(string indexName, string[] ids, float[][] vectors) { var request = new PineconeUpsertRequest(ids, vectors); var json = JsonSerializer.Serialize(request); var http = new HttpClient(); http.DefaultRequestHeaders.Add("Api-Key", "YOUR_PINECONE_KEY"); var uri = $"https://{indexName}.svc.{"YOUR_PINECONE_ENV"}.pinecone.io/vectors/upsert"; var content = new StringContent(json, Encoding.UTF8, "application/json"); var response = await http.PostAsync(uri, content); response.EnsureSuccessStatusCode(); }

Querying follows a similar pattern, but you send a single query vector and receive the nearest IDs:

public async Task<string[]> QueryAsync(string indexName, float[] queryVector, int topK = 5) { var payload = new { vector = queryVector, topK = topK, includeMetadata = false }; var json = JsonSerializer.Serialize(payload); var http = new HttpClient(); http.DefaultRequestHeaders.Add("Api-Key", "YOUR_PINECONE_KEY"); var uri = $"https://{indexName}.svc.{"YOUR_PINECONE_ENV"}.pinecone.io/query"; var content = new StringContent(json, Encoding.UTF8, "application/json"); var response = await http.PostAsync(uri, content); response.EnsureSuccessStatusCode(); var body = await response.Content.ReadAsStringAsync(); var result = JsonSerializer.Deserialize<PineconeQueryResponse>(body); return result?.matches ?? Array.Empty<string>(); }

Both methods are async, fit naturally into the .NET 8 minimal API pipeline, and can be wrapped in a service class for clean architecture.

Putting it together: a complete RAG pipeline

The core idea of RAG is three steps: embed the user query, retrieve relevant documents from the vector store, and feed the retrieved passages to the LLM as context. Below is a concise implementation that demonstrates this flow.

app.MapPost("/rag", async (OpenAIClient client, HttpContext ctx, string userQuery) => { // 1. Generate embedding using Azure OpenAI var embedResponse = await client.GetEmbeddingsAsync("text-embedding-ada-002", new EmbeddingsOptions(userQuery)); var queryVector = embedResponse.Value.Data[0].Embedding.ToArray(); // 2. Retrieve top‑3 IDs from Pinecone var ids = await QueryAsync("my-index", queryVector, 3); // 3. Fetch raw documents (simulated) var docs = ids.Select(id => $"Document content for {id}").ToArray(); // 4. Build prompt with context var prompt = $"Answer the question using only the following excerpts:\n\n{string.Join("\n\n", docs)}\n\nQuestion: {userQuery}"; // 5. Call LLM var completion = await client.GetCompletionsAsync("gpt-35-turbo", new CompletionsOptions { Prompt = { prompt }, MaxTokens = 300 }); return Results.Ok(completion.Value.Choices[0].Text.Trim()); });

This endpoint can be hit with a JSON payload { "userQuery": "How does RAG improve customer support?" } and will return a concise answer enriched by the most relevant documents stored in Pinecone.

Performance tips and monitoring

Even though Azure OpenAI handles inference, latency can still exceed 1.2 seconds if you wait for both embedding and completion sequentially. Parallelize the two calls when possible, or cache frequently used embeddings in a distributed memory store such as Azure Cache for Redis.

For vector search, Pinecone’s “metadata filtering” lets you prune results by tenant ID or document type, reducing the number of returned IDs and saving bandwidth. Enable “pod‑type” scaling only when your query volume exceeds 500 QPS to keep costs under control.

Conclusion

By leveraging .NET 8’s minimal APIs, Azure OpenAI’s robust embedding and completion endpoints, and Pinecone’s high‑performance vector search, you can build a production‑ready RAG pipeline in under an hour. The key is to keep each concern—embedding, storage, retrieval, and generation—encapsulated in its own service, allowing you to swap out components (e.g., replace Pinecone with Azure Cognitive Search) without rewriting business logic. Start with the sample code, experiment with prompt engineering, and watch your AI‑powered applications become dramatically more accurate and context‑aware.

Sources

• Azure OpenAI Service documentation – Microsoft Docs
• Pinecone Vector Database API reference – Pinecone.io
• .NET 8 release notes – dotnet.microsoft.com

Author: Mahmut Sarıkaya — sarikayadev.com