Azure Event Hubs integration

Bu içerik henüz dilinizde mevcut değil.

Azure Event Hubs is a native data-streaming service in the cloud that can stream millions of events per second, with low latency, from any source to any destination. The Aspire Azure Event Hubs integration enables you to connect to Azure Event Hubs instances from your applications.

Hosting integration

The Aspire Azure Event Hubs hosting integration models the various Event Hub resources as the following types:

AzureEventHubsResource: Represents a top-level Azure Event Hubs resource, used for representing collections of hubs and the connection information to the underlying Azure resource.
AzureEventHubResource: Represents a single Event Hub resource.
AzureEventHubsEmulatorResource: Represents an Azure Event Hubs emulator as a container resource.
AzureEventHubConsumerGroupResource: Represents a consumer group within an Event Hub resource.

To access these types and APIs for expressing them within your AppHost project, install the 📦 Aspire.Hosting.Azure.EventHubs NuGet package:

aspire add azure-eventhubs

Aspire CLI etkileşimlidir; istendiğinde uygun sonucu seçin:

Select an integration to add:

> azure-eventhubs (Aspire.Hosting.Azure.EventHubs)
> Other results listed as selectable options...

#:package Aspire.Hosting.Azure.EventHubs@*

<PackageReference Include="Aspire.Hosting.Azure.EventHubs" Version="*" />

Add an Azure Event Hubs resource

To add an AzureEventHubsResource to your AppHost project, call the AddAzureEventHubs method providing a name, and then call AddHub:

var builder = DistributedApplication.CreateBuilder(args);

var eventHubs = builder.AddAzureEventHubs("event-hubs");
eventHubs.AddHub("messages");

builder.AddProject<Projects.ExampleService>()
    .WithReference(eventHubs);

// After adding all resources, run the app...

When you add an Azure Event Hubs resource to the AppHost, it exposes other useful APIs to add Event Hub resources, consumer groups, express explicit provisioning configuration, and enables the use of the Azure Event Hubs emulator. The preceding code adds an Azure Event Hubs resource named event-hubs and an Event Hub named messages to the AppHost project. The WithReference method passes the connection information to the ExampleService project.

Connect to an existing Azure Event Hubs namespace

You might have an existing Azure Event Hubs service that you want to connect to. You can chain a call to annotate that your AzureEventHubsResource is an existing resource:

var builder = DistributedApplication.CreateBuilder(args);

var existingEventHubsName = builder.AddParameter("existingEventHubsName");
var existingEventHubsResourceGroup = builder.AddParameter("existingEventHubsResourceGroup");

var eventHubs = builder.AddAzureEventHubs("event-hubs")
    .AsExisting(existingEventHubsName, existingEventHubsResourceGroup);

builder.AddProject<Projects.ExampleProject>()
    .WithReference(eventHubs);

// After adding all resources, run the app...

For more information on treating Azure Event Hubs resources as existing resources, see Use existing Azure resources.

Add Event Hub consumer group

To add a consumer group, chain a call on an IResourceBuilder<AzureEventHubsResource> to the AddConsumerGroup API:

var builder = DistributedApplication.CreateBuilder(args);

var eventHubs = builder.AddAzureEventHubs("event-hubs");
var messages = eventHubs.AddHub("messages");
messages.AddConsumerGroup("messagesConsumer");

builder.AddProject<Projects.ExampleService>()
       .WithReference(eventHubs);

// After adding all resources, run the app...

When you call AddConsumerGroup, it configures your messages Event Hub resource to have a consumer group named messagesConsumer. The consumer group is created in the Azure Event Hubs namespace that’s represented by the AzureEventHubsResource that you added earlier. For more information, see Azure Event Hubs: Consumer groups.

Add Azure Event Hubs emulator resource

The Aspire Azure Event Hubs hosting integration supports running the Event Hubs resource as an emulator locally, based on the mcr.microsoft.com/azure-messaging/eventhubs-emulator/latest container image. This is beneficial for situations where you want to run the Event Hubs resource locally for development and testing purposes, avoiding the need to provision an Azure resource or connect to an existing Azure Event Hubs server.

To run the Event Hubs resource as an emulator, call the RunAsEmulator method:

var builder = DistributedApplication.CreateBuilder(args);

var eventHubs = builder.AddAzureEventHubs("event-hubs")
    .RunAsEmulator();

eventHubs.AddHub("messages");

var exampleProject = builder.AddProject<Projects.ExampleProject>()
    .WithReference(eventHubs);

// After adding all resources, run the app...

The preceding code configures an Azure Event Hubs resource to run locally in a container. For more information, see Azure Event Hubs Emulator.

Configure Event Hubs emulator container

There are various configurations available for container resources, for example, you can configure the container’s ports, data bind mounts, data volumes, or providing a wholistic JSON configuration which overrides everything.

Configure Event Hubs emulator container host port

By default, the Event Hubs emulator container when configured by Aspire exposes the following endpoints:

Endpoint	Image	Container port	Host port
`emulator`	`mcr.microsoft.com/azure-messaging/eventhubs-emulator/latest`	5672	dynamic

The port that it’s listening on is dynamic by default. When the container starts, the port is mapped to a random port on the host machine. To configure the endpoint port, chain calls on the container resource builder provided by the RunAsEmulator method and then the WithHostPort method as shown in the following example:

var builder = DistributedApplication.CreateBuilder(args);

var eventHubs = builder.AddAzureEventHubs("event-hubs")
    .RunAsEmulator(emulator =>
    {
        emulator.WithHostPort(7777);
    });

eventHubs.AddHub("messages");

builder.AddProject<Projects.ExampleService>()
    .WithReference(eventHubs);

// After adding all resources, run the app...

The preceding code configures the Azure Event emulator container’s existing emulator endpoint to listen on port 7777. The Azure Event emulator container’s port is mapped to the host port as shown in the following table:

Endpoint name	Port mapping (`container:host`)
`emulator`	`5672:7777`

Add Event Hubs emulator with data volume

To add a data volume to the Event Hubs emulator resource, call the WithDataVolume method on the Event Hubs emulator resource:

var builder = DistributedApplication.CreateBuilder(args);

var eventHubs = builder.AddAzureEventHubs("event-hubs")
    .RunAsEmulator(emulator =>
    {
        emulator.WithDataVolume();
    });

eventHubs.AddHub("messages");

builder.AddProject<Projects.ExampleService>()
    .WithReference(eventHubs);

// After adding all resources, run the app...

The data volume is used to persist the Event Hubs emulator data outside the lifecycle of its container. The data volume is mounted at the /data path in the container. A name is generated at random unless you provide a set the name parameter. For more information on data volumes and details on why they’re preferred over bind mounts, see Docker docs: Volumes.

Add Event Hubs emulator with data bind mount

The add a bind mount to the Event Hubs emulator container, chain a call to the WithDataBindMount API, as shown in the following example:

var builder = DistributedApplication.CreateBuilder(args);

var eventHubs = builder.AddAzureEventHubs("event-hubs")
    .RunAsEmulator(emulator =>
    {
        emulator.WithDataBindMount("/path/to/data");
    });

eventHubs.AddHub("messages");

builder.AddProject<Projects.ExampleService>()
    .WithReference(eventHubs);

// After adding all resources, run the app...

Data bind mounts rely on the host machine’s filesystem to persist the Azure Event Hubs emulator resource data across container restarts. The data bind mount is mounted at the /path/to/data path on the host machine in the container. For more information on data bind mounts, see Docker docs: Bind mounts.

Configure Event Hubs emulator container JSON configuration

The Event Hubs emulator container runs with a default config.json file. You can override this file entirely, or update the JSON configuration with a JsonNode representation of the configuration.

To provide a custom JSON configuration file, call the WithConfigurationFile method:

var builder = DistributedApplication.CreateBuilder(args);

var eventHubs = builder.AddAzureEventHubs("event-hubs")
  .RunAsEmulator(emulator =>
  {
      emulator.WithConfigurationFile("./messaging/custom-config.json");
  });

eventHubs.AddHub("messages");

builder.AddProject<Projects.ExampleService>()
    .WithReference(eventHubs);

// After adding all resources, run the app...

The preceding code configures the Event Hubs emulator container to use a custom JSON configuration file located at ./messaging/custom-config.json. This will be mounted at the /Eventhubs_Emulator/ConfigFiles/Config.json path on the container, as a read-only file. To instead override specific properties in the default configuration, call the WithConfiguration method:

var builder = DistributedApplication.CreateBuilder(args);

var eventHubs = builder.AddAzureEventHubs("event-hubs")
    .RunAsEmulator(emulator =>
    {
        emulator.WithConfiguration(
            (JsonNode configuration) =>
            {
                var userConfig = configuration["UserConfig"];
                var ns = userConfig["NamespaceConfig"][0];
                var firstEntity = ns["Entities"][0];

                firstEntity["PartitionCount"] = 5;
            });
    });

eventHubs.AddHub("messages");

builder.AddProject<Projects.ExampleService>()
    .WithReference(eventHubs);

// After adding all resources, run the app...

The preceding code retrieves the UserConfig node from the default configuration. It then updates the first entity’s PartitionCount to 5.

Provisioning-generated Bicep

If you’re new to Bicep, it’s a domain-specific language for defining Azure resources. With Aspire, you don’t need to write Bicep by-hand; instead, the provisioning APIs generate Bicep for you. When you publish your app, the generated Bicep is output alongside the manifest file. When you add an Azure Event Hubs resource, the following Bicep is generated:

@description('The location for the resource(s) to be deployed.')
param location string = resourceGroup().location

param sku string = 'Standard'

resource event_hubs 'Microsoft.EventHub/namespaces@2024-01-01' = {
  name: take('event_hubs-${uniqueString(resourceGroup().id)}', 256)
  location: location
  properties: {
    disableLocalAuth: true
  }
  sku: {
    name: sku
  }
  tags: {
    'aspire-resource-name': 'event-hubs'
  }
}

resource messages 'Microsoft.EventHub/namespaces/eventhubs@2024-01-01' = {
  name: 'messages'
  parent: event_hubs
}

output eventHubsEndpoint string = event_hubs.properties.serviceBusEndpoint

output name string = event_hubs.name

The preceding Bicep is a module that provisions an Azure Event Hubs resource. Additionally, role assignments are created for the Azure resource in a separate module:

@description('The location for the resource(s) to be deployed.')
param location string = resourceGroup().location

param event_hubs_outputs_name string

param principalType string

param principalId string

resource event_hubs 'Microsoft.EventHub/namespaces@2024-01-01' existing = {
  name: event_hubs_outputs_name
}

resource event_hubs_AzureEventHubsDataOwner 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(event_hubs.id, principalId, subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'f526a384-b230-433a-b45c-95f59c4a2dec'))
  properties: {
    principalId: principalId
    roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'f526a384-b230-433a-b45c-95f59c4a2dec')
    principalType: principalType
  }
  scope: event_hubs
}

The generated Bicep is a starting point and is influenced by changes to the provisioning infrastructure in C#. Customizations to the Bicep file directly will be overwritten, so make changes through the C# provisioning APIs to ensure they are reflected in the generated files.

Customize provisioning infrastructure

All Aspire Azure resources are subclasses of the AzureProvisioningResource type. This type enables the customization of the generated Bicep by providing a fluent API to configure the Azure resources using the ConfigureInfrastructure API. For example, you can configure the kind, consistencyPolicy, locations, and more. The following example demonstrates how to customize the Azure Event Hubs resource:

var builder = DistributedApplication.CreateBuilder(args);

builder.AddAzureEventHubs("event-hubs")
    .ConfigureInfrastructure(infra =>
    {
        var eventHubs = infra.GetProvisionableResources()
                              .OfType<EventHubsNamespace>()
                              .Single();

        eventHubs.Sku = new EventHubsSku()
        {
            Name = EventHubsSkuName.Premium,
            Tier = EventHubsSkuTier.Premium,
            Capacity = 7,
        };
        eventHubs.PublicNetworkAccess = EventHubsPublicNetworkAccess.SecuredByPerimeter;
        eventHubs.Tags.Add("ExampleKey", "Example value");
    });

The preceding code:

Chains a call to the ConfigureInfrastructure API:
- The infra parameter is an instance of the AzureResourceInfrastructure type.
- The provisionable resources are retrieved by calling the GetProvisionableResources method.
- The single EventHubsNamespace resource is retrieved.
- The Sku property is assigned to a new instance of EventHubsSku with a Premium name and tier, and a Capacity of 7.
- The PublicNetworkAccess property is assigned to SecuredByPerimeter.
- A tag is added to the Event Hubs resource with a key of ExampleKey and a value of Example value.

There are many more configuration options available to customize the Event Hubs resource. For more information, see Azure.Provisioning customization.

Hosting integration health checks

The Azure Event Hubs hosting integration automatically adds a health check for the Event Hubs resource. The health check verifies that the Event Hubs is running and that a connection can be established to it.

The hosting integration relies on the 📦 AspNetCore.HealthChecks.Azure.Messaging.EventHubs NuGet package.

Client integration

To get started with the Aspire Azure Event Hubs client integration, install the 📦 Aspire.Azure.Messaging.EventHubs NuGet package in the client-consuming project, that is, the project for the application that uses the Event Hubs client.

dotnet add package Aspire.Azure.Messaging.EventHubs

#:package Aspire.Azure.Messaging.EventHubs@*

<PackageReference Include="Aspire.Azure.Messaging.EventHubs" Version="*" />

Supported Event Hubs client types

The following Event Hub clients are supported by the library, along with their corresponding options and settings classes:

Azure client type	Azure options class	Aspire settings class
`EventHubProducerClient`	`EventHubProducerClientOptions`	`AzureMessagingEventHubsProducerSettings`
`EventHubBufferedProducerClient`	`EventHubBufferedProducerClientOptions`	`AzureMessagingEventHubsBufferedProducerSettings`
`EventHubConsumerClient`	`EventHubConsumerClientOptions`	`AzureMessagingEventHubsConsumerSettings`
`EventProcessorClient`	`EventProcessorClientOptions`	`AzureMessagingEventHubsProcessorSettings`
PartitionReceiver Class	`PartitionReceiverOptions`	`AzureMessagingEventHubsPartitionReceiverSettings`

The client types are from the Azure SDK for .NET, as are the corresponding options classes. The settings classes are provided by Aspire. The settings classes are used to configure the client instances.

Add an Event Hubs producer client

In the Program.cs file of your client-consuming project, call the AddAzureEventHubProducerClient extension method on any IHostApplicationBuilder to register an EventHubProducerClient for use via the dependency injection container. The method takes a connection name parameter.

builder.AddAzureEventHubProducerClient(connectionName: "event-hubs");

After adding the EventHubProducerClient, you can retrieve the client instance using dependency injection. For example, to retrieve your data source object from an example service define it as a constructor parameter and ensure the ExampleService class is registered with the dependency injection container:

public class ExampleService(EventHubProducerClient client)
{
    // Use client...
}

For more information, see:

Azure.Messaging.EventHubs documentation for examples on using the EventHubProducerClient.
Dependency injection in .NET for details on dependency injection.

Additional APIs to consider

The client integration provides additional APIs to configure client instances. When you need to register an Event Hubs client, consider the following APIs:

Azure client type	Registration API
`EventHubProducerClient`	`AddAzureEventHubProducerClient`
`EventHubBufferedProducerClient`	`AddAzureEventHubBufferedProducerClient`
`EventHubConsumerClient`	`AddAzureEventHubConsumerClient`
`EventProcessorClient`	`AddAzureEventProcessorClient`
`PartitionReceiver`	`AddAzurePartitionReceiverClient`

All of the aforementioned APIs include optional parameters to configure the client instances.

Add keyed Event Hubs producer client

There might be situations where you want to register multiple EventHubProducerClient instances with different connection names. To register keyed Event Hubs clients, call the AddKeyedAzureEventHubProducerClient method:

builder.AddKeyedAzureEventHubProducerClient(name: "messages");
builder.AddKeyedAzureEventHubProducerClient(name: "commands");

Then you can retrieve the client instances using dependency injection. For example, to retrieve the clients from a service:

public class ExampleService(
    [KeyedService("messages")] EventHubProducerClient messagesClient,
    [KeyedService("commands")] EventHubProducerClient commandsClient)
{
    // Use clients...
}

For more information, see Keyed services in .NET.

Additional keyed APIs to consider

The client integration provides additional APIs to configure keyed client instances. When you need to register a keyed Event Hubs client, consider the following APIs:

Azure client type	Registration API
`EventHubProducerClient`	`AddKeyedAzureEventHubProducerClient`
`EventHubBufferedProducerClient`	`AddKeyedAzureEventHubBufferedProducerClient`
`EventHubConsumerClient`	`AddKeyedAzureEventHubConsumerClient`
`EventProcessorClient`	`AddKeyedAzureEventProcessorClient`
PartitionReceiver class	`AddKeyedAzurePartitionReceiverClient`

All of the aforementioned APIs include optional parameters to configure the client instances.

Configuration

The Aspire Azure Event Hubs library provides multiple options to configure the Azure Event Hubs connection based on the requirements and conventions of your project. Either a FullyQualifiedNamespace or a ConnectionString is required to be supplied.

Use a connection string

When using a connection string from the ConnectionStrings configuration section, provide the name of the connection string when calling builder.AddAzureEventHubProducerClient() and other supported Event Hubs clients. In this example, the connection string does not include the EntityPath property, so the EventHubName property must be set in the settings callback:

builder.AddAzureEventHubProducerClient(
    "event-hubs",
    static settings =>
    {
        settings.EventHubName = "MyHub";
    });

And then the connection information will be retrieved from the ConnectionStrings configuration section. Two connection formats are supported:

Fully Qualified Namespace (FQN)

The recommended approach is to use a fully qualified namespace, which works with the Credential property to establish a connection. If no credential is configured, the DefaultAzureCredential is used.

{
  "ConnectionStrings": {
    "event-hubs": "{your_namespace}.servicebus.windows.net"
  }
}

Connection string

Alternatively, use a connection string:

{
  "ConnectionStrings": {
    "event-hubs": "Endpoint=sb://mynamespace.servicebus.windows.net/;SharedAccessKeyName=accesskeyname;SharedAccessKey=accesskey;EntityPath=MyHub"
  }
}

Use configuration providers

The Aspire Azure Event Hubs library supports Microsoft.Extensions.Configuration. It loads the AzureMessagingEventHubsSettings and the associated Options, e.g. EventProcessorClientOptions, from configuration by using the Aspire:Azure:Messaging:EventHubs: key prefix, followed by the name of the specific client in use. For example, consider the appsettings.json that configures some of the options for an EventProcessorClient:

{
  "Aspire": {
    "Azure": {
      "Messaging": {
        "EventHubs": {
          "EventProcessorClient": {
            "EventHubName": "MyHub",
            "ClientOptions": {
              "Identifier": "PROCESSOR_ID"
            }
          }
        }
      }
    }
  }
}

For the complete Azure Event Hubs client integration JSON schema, see Aspire.Azure.Messaging.EventHubs/ConfigurationSchema.json.

Use named configuration

The Aspire Azure Event Hubs library supports named configuration, which allows you to configure multiple instances of the same client type with different settings. The named configuration uses the connection name as a key under the specific client configuration section.

{
  "Aspire": {
    "Azure": {
      "Messaging": {
        "EventHubs": {
          "EventProcessorClient": {
            "processor1": {
              "EventHubName": "MyHub1",
              "ClientOptions": {
                "Identifier": "PROCESSOR_1"
              }
            },
            "processor2": {
              "EventHubName": "MyHub2",
              "ClientOptions": {
                "Identifier": "PROCESSOR_2"
              }
            }
          }
        }
      }
    }
  }
}

In this example, the processor1 and processor2 connection names can be used when calling AddAzureEventProcessorClient:

builder.AddAzureEventProcessorClient("processor1");
builder.AddAzureEventProcessorClient("processor2");

Named configuration takes precedence over the top-level configuration. If both are provided, the settings from the named configuration override the top-level settings.

You can also setup the Options type using the optional Action<IAzureClientBuilder<EventProcessorClient, EventProcessorClientOptions>> configureClientBuilder parameter of the AddAzureEventProcessorClient method. For example, to set the processor’s client ID for this client:

builder.AddAzureEventProcessorClient(
    "event-hubs",
    configureClientBuilder: clientBuilder => clientBuilder.ConfigureOptions(
        options => options.Identifier = "PROCESSOR_ID"));

Observability and telemetry

Aspire integrations automatically set up Logging, Tracing, and Metrics configurations, which are sometimes known as the pillars of observability. Depending on the backing service, some integrations may only support some of these features. For example, some integrations support logging and tracing, but not metrics. Telemetry features can also be disabled using the techniques presented in the Configuration section.

Logging

The Aspire Azure Event Hubs integration uses the following log categories:

Azure.Core
Azure.Identity

Tracing

The Aspire Azure Event Hubs integration will emit the following tracing activities using OpenTelemetry:

Azure.Messaging.EventHubs.*

Metrics

The Aspire Azure Event Hubs integration currently doesn’t support metrics by default due to limitations with the Azure SDK for .NET. If that changes in the future, this section will be updated to reflect those changes.

Sor & Yanıtla İşbirliği Topluluk Tartış İzle