Azure AI Inference integration

🧪 Preview

Azure AI Inference provides serverless API endpoints for deploying and using AI models. The Aspire Azure AI Inference integration enables you to connect to Azure AI Inference services from your applications, making it easy to call models for chat, completions, embeddings, and more.

Hosting integration

Although the Azure AI Inference library doesn’t currently offer direct hosting integration, you can still integrate it into your AppHost project. Simply add a connection string to establish a reference to an existing Azure AI Foundry resource.

Connect to an existing Azure AI Foundry service

If you already have an Azure AI Foundry service, you can easily connect to it by adding a connection string to your AppHost. This approach uses a simple, string-based configuration. To establish the connection, use the AddConnectionString method:

C# — AppHost.cs

var builder = DistributedApplication.CreateBuilder(args);

var aiFoundry = builder.AddConnectionString("ai-foundry");

builder.AddProject<Projects.ExampleProject>()
    .WithReference(aiFoundry);

// After adding all resources, run the app...

Note

Connection strings are used to represent a wide range of connection information, including database connections, message brokers, endpoint URIs, and other services. In nomenclature, the term “connection string” is used to represent any kind of connection information.

The connection string is configured in the AppHost’s configuration, typically under User Secrets, under the ConnectionStrings section:

{
  "ConnectionStrings": {
    "ai-foundry": "Endpoint=https://{endpoint}/;DeploymentId={deploymentName}"
  }
}

For more information, see Add existing Azure resources with connection strings.

Client integration

The Aspire Azure AI Inference client integration is used to work with Azure AI models using the IChatClient abstraction from Microsoft.Extensions.AI. To get started, install the 📦 Aspire.Azure.AI.Inference NuGet package.

.NET CLI

.NET CLI — Add Aspire.Azure.AI.Inference package

dotnet add package Aspire.Azure.AI.Inference

Program.cs (C# file-based app)

Program.cs

#:package Aspire.Azure.AI.Inference@*

PackageReference (*.csproj)

XML — Add Aspire.Azure.AI.Inference package reference

<PackageReference Include="Aspire.Azure.AI.Inference" Version="*" />

Add an Azure AI Inference client

In the Program.cs file of your client-consuming project, use the AddAzureChatCompletionsClient method on any IHostApplicationBuilder to register an ChatCompletionsClient for dependency injection (DI).

builder.AddAzureChatCompletionsClient(connectionName: "ai-foundry");

Tip

The connectionName parameter must match the name used when adding the Azure AI Inference resource in the AppHost project. For more information, see Connect to an existing Azure AI Foundry service.

After adding the ChatCompletionsClient, you can retrieve the client instance using dependency injection:

public class ExampleService(ChatCompletionsClient client)
{
    // Use client...
}

For more information, see:

• What is Azure AI model inference? for details on Azure AI model interfence.
• Dependency injection in .NET for details on dependency injection.
• The Azure AI Foundry SDK: C#.

Add Azure AI Inference client with registered IChatClient

If you’re interested in using the IChatClient interface with the Azure AI Inference client, simply chain either of the following APIs to the AddAzureChatCompletionsClient method:

• AddChatClient: Registers a singleton IChatClient in the services.
• AddKeyedChatClient: Registers a keyed singleton IChatClient in the services.

For example, consider the following C# code that adds an IChatClient to the DI container:

builder.AddAzureChatCompletionsClient(connectionName: "ai-foundry")
    .AddChatClient("deploymentName");

Similarly, you can add a keyed IChatClient with the following C# code:

builder.AddAzureChatCompletionsClient(connectionName: "ai-foundry")
    .AddKeyedChatClient("serviceKey", "deploymentName");

After adding the IChatClient, you can retrieve the client instance using dependency injection:

public class ExampleService(IChatClient chatClient)
{
    public async Task<string> GetResponseAsync(string userMessage)
    {
        var response = await chatClient.CompleteAsync(userMessage);
        return response.Message.Text ?? string.Empty;
    }
}

For more information on the IChatClient and its corresponding library, see Artificial intelligence in .NET.

Add keyed Azure AI Inference clients

There might be situations where you want to register multiple ChatCompletionsClient instances with different connection names. To register keyed Azure AI Inference clients, call the AddKeyedAzureChatCompletionsClient method:

builder.AddKeyedAzureChatCompletionsClient(name: "chat");
builder.AddKeyedAzureChatCompletionsClient(name: "code");

Caution

When using keyed services, ensure that your Azure AI Inference resource configures two named connections, one for chat and one for code.

Then you can retrieve the client instances using dependency injection. For example, to retrieve the clients from a service:

public class ExampleService(
    [KeyedService("chat")] ChatCompletionsClient chatClient,
    [KeyedService("code")] ChatCompletionsClient codeClient)
{
    // Use clients...
}

For more information, see Keyed services in .NET.

Configuration

The Azure AI Inference library provides multiple options to configure the Azure AI Foundry Service based on the requirements and conventions of your project.

Note

Either an Endpoint and DeploymentId, or a ConnectionString is required to be supplied.

Use a connection string

A connection can be constructed from the Keys, Deployment ID and Endpoint tab with the format:

Endpoint={endpoint};Key={key};DeploymentId={deploymentId}`

You can provide the name of the connection string when calling builder.AddAzureChatCompletionsClient():

builder.AddAzureChatCompletionsClient(
    connectionName: "connection-string-name");

The connection string is retrieved from the ConnectionStrings configuration section. Two connection formats are supported:

Azure AI Foundry endpoint

The recommended approach is to use an Endpoint, which works with the ChatCompletionsClientSettings.Credential property to establish a connection. If no credential is configured, DefaultAzureCredential is used.

{
  "ConnectionStrings": {
    "connection-string-name": "Endpoint=https://{endpoint}/;DeploymentId={deploymentName}"
  }
}

Connection string

Alternatively, a custom connection string can be used.

{
  "ConnectionStrings": {
    "connection-string-name": "Endpoint=https://{endpoint}/;Key={account_key};DeploymentId={deploymentName}"
  }
}

Use configuration providers

The Azure AI Inference library supports Microsoft.Extensions.Configuration. It loads the ChatCompletionsClientSettings and AzureAIInferenceClientOptions from configuration by using the Aspire:Azure:AI:Inference key. For example, consider an appsettings.json that configures some of the options:

{
  "Aspire": {
    "Azure": {
      "AI": {
        "Inference": {
          "DisableTracing": false,
          "EnableSensitiveTelemetryData": false,
          "ClientOptions": {
            "UserAgentApplicationId": "myapp"
          }
        }
      }
    }
  }
}

Use inline delegates

You can also pass the Action<ChatCompletionsClientSettings> configureSettings delegate to set up some or all the options inline, for example, to disable tracing from code:

builder.AddAzureChatCompletionsClient(
    connectionName: "connection-string-name",
    static settings => settings.DisableTracing = true);

Aspire integrations automatically set up Logging, Tracing, and Metrics configurations, which are sometimes known as the pillars of observability. For more information about integration observability and telemetry, see Aspire integrations overview. Depending on the backing service, some integrations may only support some of these features. For example, some integrations support logging and tracing, but not metrics. Telemetry features can also be disabled using the techniques presented in the Configuration section.

Logging

The Azure AI Inference integration uses the following log categories:

• Azure.Core
• Azure.Identity

Tracing

The Azure AI Inference integration will emit the following tracing activities using OpenTelemetry:

• Experimental.Microsoft.Extensions.AI - Used by Microsoft.Extensions.AI to record AI operations

Caution

Telemetry is only recorded by default when using the IChatClient interface from Microsoft.Extensions.AI. Raw ChatCompletionsClient calls do not automatically generate telemetry.

Configuring sensitive data in telemetry

By default, telemetry includes metadata such as token counts, but not raw inputs and outputs like message content. To include potentially sensitive information in telemetry, set the EnableSensitiveTelemetryData configuration option:

builder.AddAzureChatCompletionsClient(
    connectionName: "ai-foundry",
    configureSettings: settings =>
    {
        settings.EnableSensitiveTelemetryData = true;
    })
    .AddChatClient("deploymentName");

Or through configuration:

{
  "Aspire": {
    "Azure": {
      "AI": {
        "Inference": {
          "EnableSensitiveTelemetryData": true
        }
      }
    }
  }
}

Alternatively, you can enable sensitive data capture by setting the environment variable:

OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true

Using underlying library telemetry

If you need to access telemetry from the underlying Azure AI Inference library directly, you can manually add the appropriate activity sources and meters to your OpenTelemetry configuration:

builder.Services.AddOpenTelemetry()
    .WithTracing(tracing => tracing.AddSource("Azure.AI.Inference.*"))
    .WithMetrics(metrics => metrics.AddMeter("Azure.AI.Inference.*"));

However, you’ll need to enable experimental telemetry support in the Azure AI Inference library by setting the AZURE_EXPERIMENTAL_ENABLE_ACTIVITY_SOURCE environment variable to "true" or calling AppContext.SetSwitch("Azure.Experimental.EnableActivitySource", true) during app startup.