Running .NET on Kubernetes: A Production Guide

Q: How do I avoid dropped requests during rolling updates?

Add a preStop sleep of five to ten seconds so that endpoint removal propagates before the app receives SIGTERM . Make sure terminationGracePeriodSeconds covers the sleep plus the drain time, and combine this with maxUnavailable: 0 and a readiness probe.

Running .NET on Kubernetes in production is less about writing YAML and more about making the .NET runtime and the cluster agree on a contract: how health is reported, how much memory and CPU the process may use, how shutdown works and how the workload scales. When that contract is wrong, you get restarts under load, dropped requests during deployments and out-of-memory kills that are hard to explain. This guide is for engineers who already ship ASP.NET Core in containers. It covers deployments, probes, configuration, resource limits and their effect on the GC and thread pool, graceful shutdown, HPA and KEDA, AKS specifics, Helm and observability.

What Does Running .NET on Kubernetes Involve?#

Kubernetes runs containers in pods, keeps the desired number of replicas alive, routes traffic through Services and restarts anything that looks unhealthy. Every one of those mechanisms needs a counterpart in your application:

Platform concern	Kubernetes object or field	.NET counterpart
Running replicas and rollouts	`Deployment`	Stateless ASP.NET Core app, one image per version
Stable network identity	`Service`, Ingress or Gateway API	Kestrel on port 8080, forwarded headers
Health	Startup, readiness and liveness probes	ASP.NET Core health checks with tags
Configuration and secrets	`ConfigMap`, `Secret`, CSI volumes	Configuration providers, Key Vault, managed identity
Resources	Requests and limits	GC heap limits, `Environment.ProcessorCount`, thread pool
Shutdown	`SIGTERM`, `preStop`, grace period	`IHostApplicationLifetime`, `HostOptions.ShutdownTimeout`
Scaling	HPA, KEDA	Stateless design, fast startup, idempotent consumers

The build side is covered in the Docker best practices guide. This guide assumes you already have a small, non-root image that listens on port 8080.

How Kubernetes and the .NET Runtime Interact#

Containers are Linux processes constrained by cgroups, and the .NET runtime reads those constraints at startup. Three interactions matter most.

Memory limits drive the GC. When a container has a memory limit, the GC treats it as the machine's physical memory. By default, the heap hard limit is the larger of 20 MB and 75 percent of the limit. The remaining quarter is headroom for native memory, thread stacks, the JIT and unmanaged buffers. If you set no memory limit, the GC sizes itself against the whole node and the kernel may kill the pod when the node runs short.

CPU limits drive parallelism. Environment.ProcessorCount returns the smallest of the node's logical processors, the process affinity and the CPU limit rounded up. A limit of 1500m therefore reports 2 processors. That number sizes the Server GC heap count and the thread pool's minimum worker threads. CPU requests alone do not change it, so a pod with a request but no limit on a 32-core node reports 32 processors.

Signals drive shutdown. Kubernetes sends SIGTERM to PID 1, and the .NET host translates it into IHostApplicationLifetime.ApplicationStopping. Kestrel stops accepting new connections and in-flight requests get up to HostOptions.ShutdownTimeout, which is 30 seconds by default, to complete.

ASP.NET Core apps use Server GC by default. Since .NET 9, Server GC runs with DATAS (dynamic adaptation to application sizes) enabled. DATAS starts with a single heap and grows or shrinks the heap count with load, so heap size tracks the live data size rather than the core count. For dense clusters this is a large improvement over .NET 8, which could inflate memory on large nodes.

Getting Started: A Production-Ready Deployment and Service#

The following manifest contains the settings most teams eventually converge on. It includes a zero-downtime rolling update, a hardened security context, requests and limits, three probes, a preStop delay and a writable in-memory /tmp for a read-only root filesystem:

YAML

apiVersion: apps/v1
kind: Deployment
metadata:
  name: shop-api
  labels:
    app.kubernetes.io/name: shop-api
spec:
  replicas: 3
  selector:
    matchLabels:
      app.kubernetes.io/name: shop-api
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 0
      maxSurge: 1
  template:
    metadata:
      labels:
        app.kubernetes.io/name: shop-api
    spec:
      terminationGracePeriodSeconds: 45
      securityContext:
        runAsNonRoot: true
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: api
          image: myregistry.azurecr.io/shop-api:1.4.0
          ports:
            - name: http
              containerPort: 8080
          envFrom:
            - configMapRef:
                name: shop-api-config
          resources:
            requests:
              cpu: 500m
              memory: 512Mi
            limits:
              cpu: "2"
              memory: 512Mi
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop: ["ALL"]
          startupProbe:
            httpGet: { path: /healthz/startup, port: http }
            periodSeconds: 5
            failureThreshold: 24
          readinessProbe:
            httpGet: { path: /healthz/ready, port: http }
            periodSeconds: 10
            timeoutSeconds: 3
          livenessProbe:
            httpGet: { path: /healthz/live, port: http }
            periodSeconds: 10
            timeoutSeconds: 3
          lifecycle:
            preStop:
              sleep:
                seconds: 10
          volumeMounts:
            - name: tmp
              mountPath: /tmp
      volumes:
        - name: tmp
          emptyDir:
            medium: Memory
            sizeLimit: 64Mi
---
apiVersion: v1
kind: Service
metadata:
  name: shop-api
spec:
  selector:
    app.kubernetes.io/name: shop-api
  ports:
    - name: http
      port: 80
      targetPort: http

runAsNonRoot works with Microsoft's chiseled images because they declare a numeric user (UID 1654). maxUnavailable: 0 with maxSurge: 1 means a rollout never drops below the desired replica count. Readiness gates each new pod before an old one is removed.

Health Probes with ASP.NET Core Health Checks#

Kubernetes has three probes with different consequences, so they should call different checks:

Startup probe. It runs until it first succeeds and holds off the other probes, which protects slow warm-ups. Its budget is failureThreshold multiplied by periodSeconds, which is 120 seconds in the manifest above.
Readiness probe. A failure removes the pod from Service endpoints without restarting it. Check dependencies the pod genuinely cannot serve without, such as its primary database.
Liveness probe. A failure restarts the container. Only check whether the process itself is responsive. Never check a shared dependency here: a database outage would restart every pod at once and turn a partial outage into a total one.

Tags map health checks to probes:

using Microsoft.AspNetCore.Diagnostics.HealthChecks;
using Microsoft.Extensions.Diagnostics.HealthChecks;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddDbContext<ShopDbContext>(options =>
    options.UseNpgsql(builder.Configuration.GetConnectionString("Shop")));

var warmup = new WarmupState();
builder.Services.AddSingleton(warmup);
builder.Services.AddHostedService<CacheWarmupService>();

builder.Services.AddHealthChecks()
    .AddCheck("self", () => HealthCheckResult.Healthy(), tags: ["live"])
    .AddCheck("warmup", () => warmup.IsCompleted
        ? HealthCheckResult.Healthy()
        : HealthCheckResult.Unhealthy("Cache warm-up in progress"), tags: ["startup"])
    .AddDbContextCheck<ShopDbContext>(tags: ["ready"]);

var app = builder.Build();

app.MapHealthChecks("/healthz/startup", ForTag("startup"));
app.MapHealthChecks("/healthz/ready", ForTag("ready"));
app.MapHealthChecks("/healthz/live", ForTag("live"));

app.Run();

static HealthCheckOptions ForTag(string tag) =>
    new() { Predicate = registration => registration.Tags.Contains(tag) };

public sealed class WarmupState
{
    private volatile bool _completed;
    public bool IsCompleted => _completed;
    public void MarkCompleted() => _completed = true;
}

Keep probe endpoints cheap and bounded. The kubelet's default probe timeout is one second, and a readiness check that runs a slow query under load fails exactly when you most need capacity. Set explicit timeouts, keep dependency checks to one fast round trip, and exclude probe paths from request tracing so they do not flood your telemetry.

Configuration with ConfigMaps and Secrets#

.NET configuration maps cleanly onto Kubernetes. Environment variables from a ConfigMap or Secret arrive through the environment variables provider, where Logging__LogLevel__Default becomes Logging:LogLevel:Default. Mounted files arrive through the JSON or key-per-file providers. The two delivery modes behave differently at run time:

Environment variables are fixed for the life of the container. Changing the ConfigMap requires a rollout.
Volume-mounted ConfigMap and Secret files are updated in place after a propagation delay, unless you mount them with subPath, in which case they never update.

var builder = WebApplication.CreateBuilder(args);

// Non-secret settings mounted from a ConfigMap volume; reloads when the ConfigMap changes
builder.Configuration.AddJsonFile("/config/appsettings.k8s.json", optional: true,
    reloadOnChange: true);

// One file per key, e.g. /secrets/ConnectionStrings__Shop, from a Secret or CSI volume
// (Microsoft.Extensions.Configuration.KeyPerFile package)
builder.Configuration.AddKeyPerFile("/secrets", optional: true, reloadOnChange: true);

builder.Services.AddOptions<CheckoutOptions>()
    .Bind(builder.Configuration.GetSection("Checkout"))
    .ValidateDataAnnotations()
    .ValidateOnStart();

Kubernetes updates mounted files by atomically swapping a symlink, which file system watchers do not always detect. If reloads do not fire, set DOTNET_USE_POLLING_FILE_WATCHER=true to switch the file provider to polling. Consume reloadable values through IOptionsMonitor<T>, and treat anything that cannot change safely at run time as restart-only.

Kubernetes Secret values are only base64-encoded, so anyone allowed to read Secrets in the namespace can decode them. On AKS, prefer workload identity with Azure Key Vault. Your app can read secrets directly with DefaultAzureCredential, or the Key Vault provider for the Secrets Store CSI driver can mount them as files with rotation. The secrets management guide compares both approaches.

Resource Requests, Limits and the .NET Runtime#

Requests drive scheduling and HPA percentages. Limits drive the cgroup constraints that .NET reads. A few rules keep the two consistent:

Always set a memory limit, and usually make it equal to the request. The GC then has a hard ceiling and the pod gets predictable scheduling. If native memory is significant, for example with large gRPC buffers or native libraries, lower the GC share with DOTNET_GCHeapHardLimitPercent (hexadecimal, so 0x46 means 70 percent).
Decide deliberately about CPU limits. A limit gives .NET an accurate processor count but can cause throttling during bursts. Omitting it avoids throttling, but .NET then sizes the thread pool and GC for the whole node. If you omit limits, set DOTNET_PROCESSOR_COUNT to roughly the request, rounded up.
Prefer DATAS on .NET 9 and later. If a latency-critical service regresses, DOTNET_GCDynamicAdaptationMode=0 restores classic Server GC. For small single-core pods, consider Workstation GC with DOTNET_gcServer=0.
Size the thread pool rarely. Starvation in containers almost always comes from sync-over-async code. .NET 10 added DOTNET_ThreadPool_ForceMinWorkerThreads for the cases where a higher minimum is justified.

YAML

env:
  - name: DOTNET_GCHeapHardLimitPercent
    value: "0x46"            # 70% of the memory limit for the managed heap
  - name: DOTNET_PROCESSOR_COUNT
    value: "2"               # only when running without a CPU limit

Measure before and after any change. dotnet-counters and the runtime metrics exported by OpenTelemetry show GC heap size, pause time and thread pool queue length per pod.

Graceful Shutdown: SIGTERM, preStop and Connection Draining#

Pod termination has a well-known race. When a pod is deleted, the kubelet starts shutting it down while, at the same time, the control plane removes the pod from EndpointSlice objects. Ingress controllers and kube-proxy on other nodes learn about the removal a moment later. If your app stops listening immediately on SIGTERM, requests routed during that window fail.

The fix is to delay SIGTERM with a preStop hook. The native sleep action has been on by default since Kubernetes 1.30 and generally available since 1.34. It needs no shell, which matters for distroless images. The budget must add up: the preStop delay, plus the time the app needs to drain, must fit inside terminationGracePeriodSeconds (30 seconds by default). The manifest above gives a 10-second sleep, up to 30 seconds of draining and a 45-second grace period. Configure the host to match, and honor the stopping token in background work:

builder.Services.Configure<HostOptions>(options =>
{
    options.ShutdownTimeout = TimeSpan.FromSeconds(30);
});

public sealed class OrderConsumer(IOrderQueue queue, ILogger<OrderConsumer> logger)
    : BackgroundService
{
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            OrderMessage message;
            try
            {
                // Stop waiting for new work as soon as shutdown begins
                message = await queue.ReceiveAsync(stoppingToken);
            }
            catch (OperationCanceledException)
            {
                break;
            }

            // Let the in-flight message finish, bounded by HostOptions.ShutdownTimeout
            await queue.ProcessAndCompleteAsync(message, CancellationToken.None);
        }

        logger.LogInformation("Order consumer stopped cleanly");
    }
}

Long-running work that cannot finish within the grace period should be designed to resume: acknowledge messages only after processing and keep handlers idempotent. A PodDisruptionBudget complements this by limiting how many replicas voluntary disruptions, such as node upgrades, can take down at once.

Autoscaling with HPA and KEDA#

The Horizontal Pod Autoscaler scales on resource utilization relative to the request, so meaningful requests are a prerequisite. Use the autoscaling/v2 API. The default behavior scales up immediately, by up to 100 percent or four pods every 15 seconds. It scales down only after a five-minute stabilization window, but then it may remove every surplus replica at once. A gentler scale-down policy avoids capacity cliffs when traffic returns:

YAML

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: shop-api
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: shop-api
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 65
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 20
          periodSeconds: 60

CPU is a poor signal for queue consumers, whose backlog can grow while CPU stays flat. KEDA, a CNCF graduated project, scales on external event sources such as Azure Service Bus, Kafka, RabbitMQ or Prometheus queries, and can scale to zero. It works by feeding external metrics to an HPA it manages, and it must be the only external metrics adapter in the cluster. The current release line is 2.x, and KEDA 2.21 shipped in September 2026. This ScaledObject scales a .NET worker on queue depth and authenticates with workload identity instead of a connection string:

YAML

apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: servicebus-auth
spec:
  podIdentity:
    provider: azure-workload
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: order-consumer
spec:
  scaleTargetRef:
    name: order-consumer
  minReplicaCount: 0
  maxReplicaCount: 30
  triggers:
    - type: azure-servicebus
      metadata:
        namespace: shop-messaging
        queueName: orders
        messageCount: "50"
        activationMessageCount: "1"
      authenticationRef:
        name: servicebus-auth

messageCount is the target backlog per replica. Scale-to-zero trades cost for a cold start on the first message, so keep at least one replica for latency-sensitive consumers. On AKS, KEDA is preconfigured in AKS Automatic and available as a managed add-on in AKS Standard.

AKS Specifics: Identity, Secrets, Ingress and AKS Automatic#

On Azure Kubernetes Service, a few platform features replace work you would otherwise do yourself.

Microsoft Entra Workload ID federates a Kubernetes service account with a managed identity, so pods get Azure tokens without secrets. After you create a federated identity credential for the service account, annotate the account with the identity's client ID and label the pod template. Only labeled pods receive the injected environment variables and projected token.

YAML

apiVersion: v1
kind: ServiceAccount
metadata:
  name: shop-api
  annotations:
    azure.workload.identity/client-id: 00000000-0000-0000-0000-000000000000
---
# In the Deployment's pod template
metadata:
  labels:
    azure.workload.identity/use: "true"
spec:
  serviceAccountName: shop-api

In code, DefaultAzureCredential picks up the workload identity automatically, so the same Key Vault, Service Bus or Storage client code works locally with your developer credentials and in the cluster with the federated identity.

Ingress is changing. The Kubernetes community retired the Ingress NGINX project, and its maintenance ended in March 2026. AKS continues to provide critical security patches for the NGINX-based application routing add-on through November 2026, and it recommends Gateway API as the long-term standard. The options are the application routing Gateway API implementation, Application Gateway for Containers, or Gateway API with the Istio add-on. New .NET services should target Gateway API. Whatever you choose, enable forwarded headers, for example with ASPNETCORE_FORWARDEDHEADERS_ENABLED=true, so ASP.NET Core sees the original scheme and client IP.

AKS Automatic is now Microsoft's recommended default for most production workloads. It manages node pools, enables hardened defaults, preconfigures KEDA and turns on managed Prometheus and Container insights by default. It also carries a pod readiness SLA covering 99.9 percent of qualifying pod readiness operations within five minutes. Choose AKS Standard when you need full control over node pools, networking or add-ons.

Packaging with Helm#

Raw manifests become repetitive across environments, so most teams package services as Helm charts: templates plus a values.yaml for each environment. Helm 4 is the current major version, with Helm 3 still receiving maintenance releases. Helm 4 renamed --atomic to --rollback-on-failure, adds wait strategies and uses server-side apply for new releases.

Bash

helm upgrade --install shop-api ./charts/shop-api \
  --namespace shop --create-namespace \
  -f charts/shop-api/values.production.yaml \
  --set image.tag=1.4.0 \
  --wait --rollback-on-failure --timeout 10m

Keep charts boring. Template the image tag, replica count, resources, probe paths and environment-specific configuration, and hard-code everything else. Run helm lint and helm template in CI, and deploy the same chart version to every environment. If you use .NET Aspire, aspire publish can generate a starting chart from your AppHost; see the .NET Aspire guide.

Observability for .NET on Kubernetes#

Instrument with OpenTelemetry and export OTLP to a collector in the cluster, rather than coupling every pod to a vendor backend. Use the downward API to attach Kubernetes identity to every signal, so you can correlate a slow trace with the pod and node that produced it:

YAML

env:
  - name: POD_NAME
    valueFrom:
      fieldRef:
        fieldPath: metadata.name
  - name: OTEL_SERVICE_NAME
    value: shop-api
  - name: OTEL_EXPORTER_OTLP_ENDPOINT
    value: http://otel-collector.observability:4317
  - name: OTEL_RESOURCE_ATTRIBUTES
    value: k8s.pod.name=$(POD_NAME),k8s.namespace.name=shop

Write logs to standard output as structured JSON and let the node agent collect them. On AKS, Container insights and managed Prometheus cover platform metrics, and the Azure Monitor OpenTelemetry distro covers application telemetry. For production diagnostics, the mcr.microsoft.com/dotnet/monitor image runs as a sidecar and captures dumps and traces on demand or automatically, so you never need diagnostic tools inside your distroless image. See the OpenTelemetry guide for instrumentation details.

Best Practices#

Separate startup, readiness and liveness checks. Liveness never depends on shared infrastructure.
Set memory requests equal to limits, and keep the GC heap below the limit to leave native headroom.
Make CPU decisions explicit. Either set a limit or pin DOTNET_PROCESSOR_COUNT, so that .NET does not size itself for the whole node.
Budget shutdown end to end. The preStop sleep plus drain time must fit in terminationGracePeriodSeconds.
Run as non-root with a read-only root filesystem and dropped capabilities.
Scale on the right signal. Use CPU or RPS for APIs and queue depth through KEDA for consumers.
Use workload identity instead of secrets for Azure access, and Key Vault for the secrets that remain.
Plan the move to Gateway API now that Ingress NGINX is retired.
Add a PodDisruptionBudget and topology spread constraints so upgrades and zone failures never take down every replica.

Common Pitfalls#

OOMKilled pods with a healthy-looking GC. Native memory and thread stacks are outside the managed heap. Lower the heap percentage or raise the limit.
CPU throttling that looks like slow code. Latency spikes at a steady load often mean the CPU limit is too tight for bursts, especially during JIT and startup.
Restart storms. A liveness probe that checks the database restarts every pod during a database blip.
502 errors during every deployment. A missing preStop delay, or a grace period shorter than the drain time, drops in-flight requests.
Configuration that never reloads. Files mounted with subPath and values injected as environment variables do not update.
HPA that never scales. Without CPU requests, utilization cannot be calculated.
Assuming port 80. .NET 8 and later images listen on 8080, so container ports and probes must match.

Kubernetes vs Azure Container Apps vs App Service for .NET#

Criterion	AKS	Azure Container Apps	Azure App Service
Control over the platform	Full Kubernetes API, add-ons, node pools	Opinionated, no Kubernetes API access	Opinionated PaaS
Autoscaling	HPA, KEDA, cluster and node autoscaling	Built-in KEDA-based rules, scale to zero	Rule-based and automatic scaling
Operational burden	Highest, even with AKS Automatic	Low	Lowest
Best fit	Many services, platform teams, custom networking	Microservices and event-driven workers	Web apps and APIs with simple topologies

If you are unsure whether you need Kubernetes at all, read App Service vs Container Apps vs AKS before committing. Many .NET teams get most of the benefit with far less operational work on Container Apps.

Frequently Asked Questions#

How does .NET respect Kubernetes memory limits?#

The runtime reads the cgroup memory limit and treats it as physical memory. By default, the GC heap hard limit is 75 percent of the container limit, with a 20 MB minimum. You can change the share with DOTNET_GCHeapHardLimitPercent, or set an absolute value with DOTNET_GCHeapHardLimit.

Should I set CPU limits for ASP.NET Core pods?#

It depends on whether you value predictability or burst capacity. CPU limits make Environment.ProcessorCount match your allocation but can throttle bursts. Without limits, set DOTNET_PROCESSOR_COUNT so that the thread pool and GC do not size themselves for the entire node.

What should liveness and readiness probes check in .NET?#

Liveness should only confirm that the process responds, for example with a trivial check tagged live. Readiness should check the critical dependencies the pod needs to serve traffic. Use a startup probe for slow warm-up instead of long initial delays.

How do I avoid dropped requests during rolling updates?#

Add a preStop sleep of five to ten seconds so that endpoint removal propagates before the app receives SIGTERM. Make sure terminationGracePeriodSeconds covers the sleep plus the drain time, and combine this with maxUnavailable: 0 and a readiness probe.

When should I use KEDA instead of the HPA?#

Use KEDA when the scaling signal lives outside the pod, such as queue depth, stream lag or a Prometheus query, or when you want scale to zero. KEDA still drives an HPA under the hood, so you get the same scaling behavior controls.

Summary#

Treat the pod spec as a contract with the .NET runtime: memory limits set the GC ceiling, CPU limits set the processor count, and SIGTERM starts host shutdown.
Use distinct startup, readiness and liveness checks, and keep them fast.
Drain gracefully with a native preStop sleep and a grace period that covers your shutdown timeout.
Scale APIs with the HPA and event consumers with KEDA, using workload identity rather than connection strings.
On AKS, start with AKS Automatic, move ingress to Gateway API and package with Helm 4.

What Does Running .NET on Kubernetes Involve?#

How Kubernetes and the .NET Runtime Interact#

Getting Started: A Production-Ready Deployment and Service#

Health Probes with ASP.NET Core Health Checks#

Configuration with ConfigMaps and Secrets#

Resource Requests, Limits and the .NET Runtime#

Graceful Shutdown: SIGTERM, preStop and Connection Draining#

Autoscaling with HPA and KEDA#

AKS Specifics: Identity, Secrets, Ingress and AKS Automatic#

Packaging with Helm#

Observability for .NET on Kubernetes#

Best Practices#

Common Pitfalls#

Kubernetes vs Azure Container Apps vs App Service for .NET#

Frequently Asked Questions#

How does .NET respect Kubernetes memory limits?#

Should I set CPU limits for ASP.NET Core pods?#

What should liveness and readiness probes check in .NET?#

How do I avoid dropped requests during rolling updates?#

When should I use KEDA instead of the HPA?#

Summary#

Further Reading#

Related articles