Configure Ingress on AKS
Esta página aún no está disponible en tu idioma.
This walkthrough shows how to use AddIngress to expose .NET Aspire services on
Azure Kubernetes Service (AKS) with Application Gateway for Containers (AGC)
and automatic TLS certificate provisioning via cert-manager DNS-01 challenges.
Set up the AKS environment
Section titled “Set up the AKS environment”The following steps create all the Azure infrastructure needed for this walkthrough. If you already have an AKS cluster with ALB and cert-manager configured, skip to Configure the AppHost.
Register resource providers and features
Section titled “Register resource providers and features”Application Gateway for Containers requires several resource providers and preview features to be registered in your Azure subscription:
# Register required resource providersaz provider register --namespace Microsoft.ContainerServiceaz provider register --namespace Microsoft.Networkaz provider register --namespace Microsoft.NetworkFunctionaz provider register --namespace Microsoft.ServiceNetworking
# Register preview features for ALB controller and Gateway APIaz feature register --namespace Microsoft.ContainerService \ --name ManagedGatewayAPIPreviewaz feature register --namespace Microsoft.ContainerService \ --name ApplicationLoadBalancerPreview
# Wait for registration to complete (may take a few minutes)az feature show --namespace Microsoft.ContainerService \ --name ApplicationLoadBalancerPreview --query properties.state -o tsv
# Re-register the provider after feature registrationaz provider register --namespace Microsoft.ContainerService
# Install required CLI extensionsaz extension add --name albaz extension add --name aks-preview# Register required resource providersaz provider register --namespace Microsoft.ContainerServiceaz provider register --namespace Microsoft.Networkaz provider register --namespace Microsoft.NetworkFunctionaz provider register --namespace Microsoft.ServiceNetworking
# Register preview features for ALB controller and Gateway APIaz feature register --namespace Microsoft.ContainerService ` --name ManagedGatewayAPIPreviewaz feature register --namespace Microsoft.ContainerService ` --name ApplicationLoadBalancerPreview
# Wait for registration to complete (may take a few minutes)az feature show --namespace Microsoft.ContainerService ` --name ApplicationLoadBalancerPreview --query properties.state -o tsv
# Re-register the provider after feature registrationaz provider register --namespace Microsoft.ContainerService
# Install required CLI extensionsaz extension add --name albaz extension add --name aks-previewCreate the resource group and cluster
Section titled “Create the resource group and cluster”# Variables — customize these for your environmentRESOURCE_GROUP=my-aspire-aksLOCATION=westus3CLUSTER_NAME=my-aspire-aksACR_NAME=myaspireacrDNS_ZONE=myapp.example.com
# Create resource groupaz group create --name $RESOURCE_GROUP --location $LOCATION
# Create AKS cluster with ALB controller, OIDC issuer, and workload identityaz aks create \ --resource-group $RESOURCE_GROUP \ --name $CLUSTER_NAME \ --location $LOCATION \ --enable-oidc-issuer \ --enable-workload-identity \ --generate-ssh-keys
# Enable the ALB controller addonaz aks update \ --resource-group $RESOURCE_GROUP \ --name $CLUSTER_NAME \ --enable-alb
# Get cluster credentialsaz aks get-credentials \ --resource-group $RESOURCE_GROUP \ --name $CLUSTER_NAME# Variables — customize these for your environment$RESOURCE_GROUP = "my-aspire-aks"$LOCATION = "westus3"$CLUSTER_NAME = "my-aspire-aks"$ACR_NAME = "myaspireacr"$DNS_ZONE = "myapp.example.com"
# Create resource groupaz group create --name $RESOURCE_GROUP --location $LOCATION
# Create AKS cluster with ALB controller, OIDC issuer, and workload identityaz aks create ` --resource-group $RESOURCE_GROUP ` --name $CLUSTER_NAME ` --location $LOCATION ` --enable-oidc-issuer ` --enable-workload-identity ` --generate-ssh-keys
# Enable the ALB controller addonaz aks update ` --resource-group $RESOURCE_GROUP ` --name $CLUSTER_NAME ` --enable-alb
# Get cluster credentialsaz aks get-credentials ` --resource-group $RESOURCE_GROUP ` --name $CLUSTER_NAMECreate and attach a container registry
Section titled “Create and attach a container registry”# Create Azure Container Registryaz acr create \ --resource-group $RESOURCE_GROUP \ --name $ACR_NAME \ --sku Basic
# Attach ACR to AKS (grants AcrPull access)az aks update \ --resource-group $RESOURCE_GROUP \ --name $CLUSTER_NAME \ --attach-acr $ACR_NAME# Create Azure Container Registryaz acr create ` --resource-group $RESOURCE_GROUP ` --name $ACR_NAME ` --sku Basic
# Attach ACR to AKS (grants AcrPull access)az aks update ` --resource-group $RESOURCE_GROUP ` --name $CLUSTER_NAME ` --attach-acr $ACR_NAMECreate the ApplicationLoadBalancer
Section titled “Create the ApplicationLoadBalancer”Create a dedicated subnet for AGC in the cluster’s VNet, then deploy the ApplicationLoadBalancer custom resource:
# Get the managed cluster resource group and VNetMC_RG=$(az aks show --resource-group $RESOURCE_GROUP --name $CLUSTER_NAME \ --query nodeResourceGroup -o tsv)VNET_NAME=$(az network vnet list --resource-group $MC_RG --query "[0].name" -o tsv)
# Create a subnet for the ALB (delegated to ServiceNetworking)az network vnet subnet create \ --resource-group $MC_RG \ --vnet-name $VNET_NAME \ --name subnet-alb \ --address-prefix 10.237.0.0/24 \ --delegations Microsoft.ServiceNetworking/trafficControllers
# Get the full subnet IDSUBNET_ID=$(az network vnet subnet show \ --resource-group $MC_RG \ --vnet-name $VNET_NAME \ --name subnet-alb \ --query id -o tsv)
# Deploy the ApplicationLoadBalancer CRDkubectl apply -f - <<EOFapiVersion: alb.networking.azure.io/v1kind: ApplicationLoadBalancermetadata: name: alb-aspire namespace: defaultspec: associations: - $SUBNET_IDEOF
# Wait for it to become readykubectl get applicationloadbalancer alb-aspire --watch# Get the managed cluster resource group and VNet$MC_RG = $(az aks show --resource-group $RESOURCE_GROUP --name $CLUSTER_NAME ` --query nodeResourceGroup -o tsv)$VNET_NAME = $(az network vnet list --resource-group $MC_RG --query "[0].name" -o tsv)
# Create a subnet for the ALB (delegated to ServiceNetworking)az network vnet subnet create ` --resource-group $MC_RG ` --vnet-name $VNET_NAME ` --name subnet-alb ` --address-prefix 10.237.0.0/24 ` --delegations Microsoft.ServiceNetworking/trafficControllers
# Get the full subnet ID$SUBNET_ID = $(az network vnet subnet show ` --resource-group $MC_RG ` --vnet-name $VNET_NAME ` --name subnet-alb ` --query id -o tsv)
# Deploy the ApplicationLoadBalancer CRD@"apiVersion: alb.networking.azure.io/v1kind: ApplicationLoadBalancermetadata: name: alb-aspire namespace: defaultspec: associations: - $SUBNET_ID"@ | kubectl apply -f -
# Wait for it to become readykubectl get applicationloadbalancer alb-aspire --watchCreate an Azure DNS zone
Section titled “Create an Azure DNS zone”# Create the DNS zoneaz network dns zone create \ --resource-group $RESOURCE_GROUP \ --name $DNS_ZONE
# List the name servers — delegate these in your domain registraraz network dns zone show \ --resource-group $RESOURCE_GROUP \ --name $DNS_ZONE \ --query nameServers -o tsv# Create the DNS zoneaz network dns zone create ` --resource-group $RESOURCE_GROUP ` --name $DNS_ZONE
# List the name servers — delegate these in your domain registraraz network dns zone show ` --resource-group $RESOURCE_GROUP ` --name $DNS_ZONE ` --query nameServers -o tsvInstall cert-manager with Azure DNS support
Section titled “Install cert-manager with Azure DNS support”# Install cert-manager with Gateway API supporthelm upgrade --install cert-manager oci://quay.io/jetstack/charts/cert-manager \ --namespace cert-manager \ --create-namespace \ --set crds.enabled=true \ --set config.enableGatewayAPI=true \ --wait
# Create a managed identity for cert-manageraz identity create \ --name cert-manager-identity \ --resource-group $RESOURCE_GROUP
IDENTITY_CLIENT_ID=$(az identity show \ --name cert-manager-identity \ --resource-group $RESOURCE_GROUP \ --query clientId -o tsv)
IDENTITY_PRINCIPAL_ID=$(az identity show \ --name cert-manager-identity \ --resource-group $RESOURCE_GROUP \ --query principalId -o tsv)
# Grant DNS Zone Contributor role on the DNS zoneDNS_ZONE_ID=$(az network dns zone show \ --resource-group $RESOURCE_GROUP \ --name $DNS_ZONE \ --query id -o tsv)
az role assignment create \ --assignee-object-id $IDENTITY_PRINCIPAL_ID \ --assignee-principal-type ServicePrincipal \ --role "DNS Zone Contributor" \ --scope $DNS_ZONE_ID
# Create a federated credential for cert-manager's service accountOIDC_ISSUER=$(az aks show \ --resource-group $RESOURCE_GROUP \ --name $CLUSTER_NAME \ --query oidcIssuerProfile.issuerUrl -o tsv)
az identity federated-credential create \ --name cert-manager-fedcred \ --identity-name cert-manager-identity \ --resource-group $RESOURCE_GROUP \ --issuer $OIDC_ISSUER \ --subject "system:serviceaccount:cert-manager:cert-manager" \ --audiences "api://AzureADTokenExchange"
# Annotate and label the cert-manager service account for workload identitykubectl annotate serviceaccount cert-manager \ --namespace cert-manager \ azure.workload.identity/client-id=$IDENTITY_CLIENT_ID \ --overwrite
kubectl label serviceaccount cert-manager \ --namespace cert-manager \ azure.workload.identity/use=true \ --overwrite
# Patch the deployment to inject workload identity env varskubectl patch deployment cert-manager \ --namespace cert-manager \ --type=json \ -p='[{"op":"add","path":"/spec/template/metadata/labels/azure.workload.identity~1use","value":"true"}]'
# Wait for rolloutkubectl rollout status deployment cert-manager --namespace cert-manager
# Get the subscription ID for the ClusterIssuerSUBSCRIPTION_ID=$(az account show --query id -o tsv)
# Create a ClusterIssuer using Azure DNS for DNS-01 challengeskubectl apply -f - <<EOFapiVersion: cert-manager.io/v1kind: ClusterIssuermetadata: name: letsencrypt-prodspec: acme: server: https://acme-v02.api.letsencrypt.org/directory email: your-email@example.com privateKeySecretRef: name: letsencrypt-prod-key solvers: - dns01: azureDNS: hostedZoneName: $DNS_ZONE resourceGroupName: $RESOURCE_GROUP subscriptionID: $SUBSCRIPTION_ID environment: AzurePublicCloud managedIdentity: clientID: $IDENTITY_CLIENT_IDEOF
# Verify the issuer is readykubectl get clusterissuer letsencrypt-prod# Install cert-manager with Gateway API supporthelm upgrade --install cert-manager oci://quay.io/jetstack/charts/cert-manager ` --namespace cert-manager ` --create-namespace ` --set crds.enabled=true ` --set config.enableGatewayAPI=true ` --wait
# Create a managed identity for cert-manageraz identity create ` --name cert-manager-identity ` --resource-group $RESOURCE_GROUP
$IDENTITY_CLIENT_ID = $(az identity show ` --name cert-manager-identity ` --resource-group $RESOURCE_GROUP ` --query clientId -o tsv)
$IDENTITY_PRINCIPAL_ID = $(az identity show ` --name cert-manager-identity ` --resource-group $RESOURCE_GROUP ` --query principalId -o tsv)
# Grant DNS Zone Contributor role on the DNS zone$DNS_ZONE_ID = $(az network dns zone show ` --resource-group $RESOURCE_GROUP ` --name $DNS_ZONE ` --query id -o tsv)
az role assignment create ` --assignee-object-id $IDENTITY_PRINCIPAL_ID ` --assignee-principal-type ServicePrincipal ` --role "DNS Zone Contributor" ` --scope $DNS_ZONE_ID
# Create a federated credential for cert-manager's service account$OIDC_ISSUER = $(az aks show ` --resource-group $RESOURCE_GROUP ` --name $CLUSTER_NAME ` --query oidcIssuerProfile.issuerUrl -o tsv)
az identity federated-credential create ` --name cert-manager-fedcred ` --identity-name cert-manager-identity ` --resource-group $RESOURCE_GROUP ` --issuer $OIDC_ISSUER ` --subject "system:serviceaccount:cert-manager:cert-manager" ` --audiences "api://AzureADTokenExchange"
# Annotate and label the cert-manager service account for workload identitykubectl annotate serviceaccount cert-manager ` --namespace cert-manager ` azure.workload.identity/client-id=$IDENTITY_CLIENT_ID ` --overwrite
kubectl label serviceaccount cert-manager ` --namespace cert-manager ` azure.workload.identity/use=true ` --overwrite
# Patch the deployment to inject workload identity env varskubectl patch deployment cert-manager ` --namespace cert-manager ` --type=json ` -p='[{"op":"add","path":"/spec/template/metadata/labels/azure.workload.identity~1use","value":"true"}]'
# Wait for rolloutkubectl rollout status deployment cert-manager --namespace cert-manager
# Get the subscription ID for the ClusterIssuer$SUBSCRIPTION_ID = $(az account show --query id -o tsv)
# Create a ClusterIssuer using Azure DNS for DNS-01 challenges@"apiVersion: cert-manager.io/v1kind: ClusterIssuermetadata: name: letsencrypt-prodspec: acme: server: https://acme-v02.api.letsencrypt.org/directory email: your-email@example.com privateKeySecretRef: name: letsencrypt-prod-key solvers: - dns01: azureDNS: hostedZoneName: $DNS_ZONE resourceGroupName: $RESOURCE_GROUP subscriptionID: $SUBSCRIPTION_ID environment: AzurePublicCloud managedIdentity: clientID: $IDENTITY_CLIENT_ID"@ | kubectl apply -f -
# Verify the issuer is readykubectl get clusterissuer letsencrypt-prodConfigure the AppHost
Section titled “Configure the AppHost”Define your Kubernetes environment, container registry, and ingress in the AppHost project. The example below uses parameters so that environment-specific values are supplied at deploy time.
var builder = DistributedApplication.CreateBuilder(args);
// Parametersvar registryEndpoint = builder.AddParameter("registryEndpoint");var helmNamespace = builder.AddParameter("helmNamespace");var helmChartVersion = builder.AddParameter("helmChartVersion");var helmChartReleaseName = builder.AddParameter("helmChartReleaseName");var ingressClassName = builder.AddParameter("ingressClassName");var loadBalancerName = builder.AddParameter("loadBalancerName");var loadBalancerNamespace = builder.AddParameter("loadBalancerNamespace");var clusterIssuer = builder.AddParameter("clusterIssuer");var externalFqdn = builder.AddParameter("externalFqdn");
// Kubernetes environmentvar k8s = builder.AddKubernetesEnvironment("k8s") .WithHelm(helm => { helm.Namespace = helmNamespace; helm.ChartVersion = helmChartVersion; helm.ReleaseName = helmChartReleaseName; });
// Container registryvar acr = builder.AddContainerRegistry("acr", registryEndpoint);
// Application servicesvar api = builder.AddProject<Projects.ApiService>("api");
var frontend = builder.AddProject<Projects.Frontend>("frontend") .WithReference(api);
// Ingressvar ingress = k8s.AddIngress("ingress") .WithIngressClass(ingressClassName) .WithIngressAnnotation( "alb.networking.azure.io/alb-name", loadBalancerName) .WithIngressAnnotation( "alb.networking.azure.io/alb-namespace", loadBalancerNamespace) .WithIngressAnnotation( "cert-manager.io/cluster-issuer", clusterIssuer) .WithDefaultBackend(frontend.GetEndpoint("http")) .WithHostname(externalFqdn) .WithTls();
builder.Build().Run();import { DistributedApplication } from "@aspire/apphost";
const builder = DistributedApplication.createBuilder();
// Parametersconst registryEndpoint = builder.addParameter("registryEndpoint");const helmNamespace = builder.addParameter("helmNamespace");const helmChartVersion = builder.addParameter("helmChartVersion");const helmChartReleaseName = builder.addParameter("helmChartReleaseName");const ingressClassName = builder.addParameter("ingressClassName");const loadBalancerName = builder.addParameter("loadBalancerName");const loadBalancerNamespace = builder.addParameter("loadBalancerNamespace");const clusterIssuer = builder.addParameter("clusterIssuer");const externalFqdn = builder.addParameter("externalFqdn");
// Kubernetes environmentconst k8s = builder.addKubernetesEnvironment("k8s") .withHelm((helm) => { helm.namespace = helmNamespace; helm.chartVersion = helmChartVersion; helm.releaseName = helmChartReleaseName; });
// Container registryconst acr = builder.addContainerRegistry("acr", registryEndpoint);
// Application servicesconst api = builder.addProject("api");
const frontend = builder.addProject("frontend") .withReference(api);
// Ingressconst ingress = k8s.addIngress("ingress") .withIngressClass(ingressClassName) .withIngressAnnotation( "alb.networking.azure.io/alb-name", loadBalancerName) .withIngressAnnotation( "alb.networking.azure.io/alb-namespace", loadBalancerNamespace) .withIngressAnnotation( "cert-manager.io/cluster-issuer", clusterIssuer) .withDefaultBackend(frontend.getEndpoint("http")) .withHostname(externalFqdn) .withTls();
builder.build().run();Deploy
Section titled “Deploy”Set environment variables to provide parameter values, then run aspire deploy:
export Parameters__registryEndpoint=myregistry.azurecr.ioexport Parameters__helmNamespace=my-appexport Parameters__helmChartVersion=0.1.0export Parameters__helmChartReleaseName=my-appexport Parameters__ingressClassName=azure-alb-externalexport Parameters__loadBalancerName=my-albexport Parameters__loadBalancerNamespace=alb-infraexport Parameters__clusterIssuer=letsencrypt-dnsexport Parameters__externalFqdn=myapp.example.com
aspire deployVerify
Section titled “Verify”After the deployment completes, verify the ingress is working:
-
Check the ingress address. Wait for AGC to assign an IP or FQDN to the ingress:
Terminal window kubectl get ingress -n my-appThe
ADDRESScolumn should show the AGC frontend address. -
Check the certificate status. Verify that cert-manager has issued a certificate for your domain:
Terminal window kubectl get certificate -n my-appThe
READYcolumn should showTrueonce the certificate is issued. -
Access the application. Open
https://myapp.example.comin your browser and confirm you see a valid TLS certificate.
How TLS works with Ingress on AGC
Section titled “How TLS works with Ingress on AGC”When you deploy an ingress with the cert-manager.io/cluster-issuer annotation
and WithTls, the following sequence occurs:
-
Self-signed bootstrap certificate. On the first deploy, cert-manager creates a temporary self-signed certificate so AGC can start serving traffic immediately.
-
Annotation detection. cert-manager detects the
cert-manager.io/cluster-issuerannotation on the ingress resource and creates aCertificateresource targeting the configuredClusterIssuer. -
DNS-01 challenge. The
ClusterIssueruses the DNS-01 solver to validate domain ownership. cert-manager creates aTXTrecord in your Azure DNS zone, the ACME server validates it, and a real certificate is issued. -
Certificate rotation. cert-manager stores the issued certificate in a Kubernetes secret referenced by the ingress TLS configuration and automatically renews it before expiry.
Deploy without TLS
Section titled “Deploy without TLS”Deploy without TLS (not recommended for production)
Remove the WithTls call and the cert-manager.io/cluster-issuer annotation:
var ingress = k8s.AddIngress("ingress") .WithIngressClass(ingressClassName) .WithIngressAnnotation( "alb.networking.azure.io/alb-name", loadBalancerName) .WithIngressAnnotation( "alb.networking.azure.io/alb-namespace", loadBalancerNamespace) .WithDefaultBackend(frontend.GetEndpoint("http")) .WithHostname(externalFqdn);const ingress = k8s.addIngress("ingress") .withIngressClass(ingressClassName) .withIngressAnnotation( "alb.networking.azure.io/alb-name", loadBalancerName) .withIngressAnnotation( "alb.networking.azure.io/alb-namespace", loadBalancerNamespace) .withDefaultBackend(frontend.getEndpoint("http")) .withHostname(externalFqdn);Deploy without hostname
Section titled “Deploy without hostname”Deploy without a custom hostname
Remove the WithHostname, WithTls, and cert-manager annotation calls:
var ingress = k8s.AddIngress("ingress") .WithIngressClass(ingressClassName) .WithIngressAnnotation( "alb.networking.azure.io/alb-name", loadBalancerName) .WithIngressAnnotation( "alb.networking.azure.io/alb-namespace", loadBalancerNamespace) .WithDefaultBackend(frontend.GetEndpoint("http"));const ingress = k8s.addIngress("ingress") .withIngressClass(ingressClassName) .withIngressAnnotation( "alb.networking.azure.io/alb-name", loadBalancerName) .withIngressAnnotation( "alb.networking.azure.io/alb-namespace", loadBalancerNamespace) .withDefaultBackend(frontend.getEndpoint("http"));Troubleshooting
Section titled “Troubleshooting”AGC frontend limit
Section titled “AGC frontend limit”AGC has a limit of 5 frontends per Application Gateway for Containers instance. Each Ingress resource creates its own frontend. If you exceed this limit, new ingress resources remain in a pending state.
To check your current frontend count:
kubectl get ingress -A -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.loadBalancer.ingress[0].hostname}{"\n"}{end}'Consider consolidating multiple services behind a single ingress with path-based routing, or request a quota increase from Azure support.
Certificate not issuing
Section titled “Certificate not issuing”If the certificate READY status remains False, check the following:
-
cert-manager logs. Look for errors related to DNS-01 challenge creation:
Terminal window kubectl logs -n cert-manager deploy/cert-manager -f -
Challenge status. Check if the ACME challenge is progressing:
Terminal window kubectl get challenges -n my-app -
Workload identity. Verify that the cert-manager service account has the correct federated credential and that the managed identity has DNS Zone Contributor permissions on the Azure DNS zone:
Terminal window kubectl describe serviceaccount cert-manager -n cert-manager -
ClusterIssuer status. Confirm the issuer is ready:
Terminal window kubectl get clusterissuer letsencrypt-dns -o yaml
Browser showing HSTS redirect to HTTPS
Section titled “Browser showing HSTS redirect to HTTPS”If your browser automatically redirects HTTP URLs to HTTPS even when you haven’t
configured TLS, this is due to HTTP Strict Transport Security (HSTS). Modern
browsers cache HSTS policies for domains, including subdomains of .com and
other top-level domains.
To work around this during development:
- Clear your browser’s HSTS cache for the domain.
- Use a private/incognito browser window.
- Deploy with TLS enabled to avoid the issue entirely.