Building Retrieval Augmented Generation APIs with .NET 8, Azure AI Search, and OpenAI

Why RAG matters for modern AI services

Ever wondered why a large language model can generate fluent text yet still hallucinate facts? Retrieval Augmented Generation (RAG) solves that problem by grounding the model’s output in a searchable knowledge base. In 2023, Microsoft reported a 30% reduction in hallucinations for enterprise bots that used Azure AI Search as a retrieval layer. For .NET developers, the combination of .NET 8 Minimal APIs, Azure AI Search, and OpenAI creates a low‑latency, scalable RAG pipeline that can be deployed in minutes.

Core concepts of Retrieval Augmented Generation

RAG works in three steps: (1) a user query is sent to a vector store, (2) the most relevant documents are fetched, and (3) the retrieved passages are injected into the prompt sent to the LLM. The approach keeps the LLM lightweight because it does not need to memorize the entire corpus. Azure AI Search provides built‑in semantic ranking and vector search, while OpenAI’s gpt‑4o‑mini model offers cost‑effective generation.

Creating a .NET 8 Minimal API project

Start with the .NET 8 SDK (released November 2023). The minimal API template reduces boilerplate to a single Program.cs file. Run the following commands in a Bash terminal:

dotnet new web -n RagApi --framework net8.0
cd RagApi
dotnet add package Azure.Search.Documents
dotnet add package OpenAI

After the packages are installed, replace Program.cs with the code shown below. The example defines a single POST endpoint /chat that accepts a JSON payload containing question.

using Azure;
using Azure.Search.Documents;
using Azure.Search.Documents.Models;
using OpenAI;
using OpenAI.Chat;
var builder = WebApplication.CreateBuilder(args);
// Configuration values (store in user‑secrets or Azure Key Vault in production)
var searchEndpoint = builder.Configuration["Search:Endpoint"]!;
var searchKey = builder.Configuration["Search:Key"]!;
var indexName = builder.Configuration["Search:IndexName"]!;
var openAiKey = builder.Configuration["OpenAI:Key"]!;
var client = builder.Build();
var searchClient = new SearchClient(new Uri(searchEndpoint), indexName, new AzureKeyCredential(searchKey));
var openAiClient = new OpenAIClient(openAiKey);
client.MapPost("/chat", async (ChatRequest request) =>
{
    // 1️⃣ Retrieve top‑3 relevant documents
    var vector = await GetEmbeddingAsync(request.Question, openAiClient);
    var options = new SearchOptions { Size = 3, IncludeTotalCount = false };
    options.VectorSearch = new VectorSearchOptions { Queries = { new VectorQuery(vector, k: 3) } };
    var response = await searchClient.SearchAsync<SearchDocument>("*", options);
    var docs = new List<string>();
    await foreach (var result in response.Value.GetResultsAsync())
    {
        docs.Add(result.Document["content"].ToString());
    }
    // 2️⃣ Build the augmented prompt
    var augmented = $"Context:\n{string.Join("\n---\n", docs)}\n\nQuestion: {request.Question}\nAnswer:";
    // 3️⃣ Call OpenAI completion
>    var chat = new ChatCompletionRequest(augmented, model: "gpt-4o-mini");
    var completion = await openAiClient.ChatEndpoint.GetCompletionAsync(chat);
    return Results.Ok(new { answer = completion.FirstChoice.Message.Content });
});
client.Run();
record struct ChatRequest(string Question);
async Task<float[]> GetEmbeddingAsync(string text, OpenAIClient client)
{
    var embed = await client.EmbeddingsEndpoint.CreateEmbeddingAsync(text, model: "text-embedding-ada-002");
    return embed.Data[0].Embedding;
}

The code demonstrates the full RAG loop: embedding generation, vector search, prompt construction, and final answer. Notice the use of await foreach to stream search results, which keeps memory usage low even for large indexes.

Provisioning Azure AI Search

Azure AI Search requires an Azure subscription and a Search service (Standard tier is sufficient for most prototypes). Create the service via Azure CLI:

az group create --name rag-demo-rg --location eastus
az search service create --name ragdemo --resource-group rag-demo-rg --location eastus --sku standard

Next, define an index that stores documents with a content field and a vector field for embeddings:

{
  "name": "documents",
  "fields": [
    {"name": "id", "type": "Edm.String", "key": true},
    {"name": "content", "type": "Edm.String", "searchable": true},
    {"name": "vector", "type": "Collection(Edm.Single)", "searchable": false, "vectorSearchDimensions": 1536}
  ]
}

Upload your knowledge base (e.g., product FAQs) using the az search document index add command or the .NET SDK. Remember to generate embeddings with the same model (text-embedding-ada-002) before indexing.

Calling OpenAI from .NET

The official OpenAI NuGet package abstracts HTTP calls. Use the ChatEndpoint for completions and the EmbeddingsEndpoint for vector creation, as shown in the minimal API code. For production, enable retry policies via Polly and respect the rate limits published by OpenAI (e.g., 60 RPM for free tier).

Testing the RAG endpoint locally

Run the API with dotnet run. A Swagger UI is automatically available at http://localhost:5080/swagger. Post a JSON body like:

{ "question": "How do I reset my password in the portal?" }

The response will include the answer generated from the most relevant documents. Use tools such as curl or Postman to automate load testing; a typical latency for a 3‑document retrieval plus OpenAI call is around 850 ms on Azure East US.

Key takeaways for .NET developers

1. .NET 8 Minimal APIs let you spin up a production‑grade RAG service in under ten minutes.
2. Azure AI Search handles vector storage, scoring, and scaling without custom infrastructure.
3. OpenAI’s embedding and chat models integrate seamlessly via the official SDK, keeping the codebase concise.
4. Secure secrets with Azure Key Vault and enable Application Insights for telemetry to monitor latency and error rates.

Conclusion

By combining .NET 8 Minimal APIs, Azure AI Search, and OpenAI, you can deliver trustworthy, context‑aware answers that scale with enterprise workloads. The pattern illustrated here—embedding, retrieval, augmentation, generation—has become the de‑facto standard for building reliable conversational experiences. Adopt the code snippets, adjust the index schema to your domain, and you’ll have a production‑ready RAG service ready for integration with web front‑ends, Teams bots, or internal portals.

Author: Mahmut Sarıkaya — sarikayadev.com

Sources

Microsoft Docs – Azure AI Search documentation
OpenAI API reference – official OpenAI documentation
.NET Blog – .NET 8 release notes