High-Throughput .NET Services Interview Questions

Q1 How do you decide Kestrel's connection and request limits for a high-throughput service, and what happens when you get them wrong?#

Short answer: Kestrel's limits exist to protect the process from being overwhelmed by more concurrent work than it can finish in a reasonable time. You size them from measured capacity (how many requests the service can serve concurrently while holding your latency SLO), not from guesswork, and you treat the reverse proxy or load balancer in front of Kestrel as the first line of defense.

KestrelServerOptions.Limits exposes the knobs that matter most in practice. MaxConcurrentConnections caps the number of open TCP connections; MaxConcurrentUpgradedConnections does the same for connections upgraded to WebSockets, which are not counted against the first limit. MaxRequestBodySize bounds how large a single request body can be before Kestrel aborts it, which protects memory and prevents slow, large uploads from tying up a connection indefinitely. MinRequestBodyDataRate and MinResponseDataRate reject connections that transfer data too slowly, which is the defense against slow-loris-style attacks and misbehaving clients that would otherwise hold a connection (and its buffers) open forever. Getting these wrong in either direction is costly: set them too low and you reject legitimate bursty traffic with connection resets or 503s; set them too high, or leave them unset, and a spike in slow or malicious clients can exhaust threads, sockets and memory before your autoscaler even reacts.

builder.WebHost.ConfigureKestrel(options =>
{
    options.Limits.MaxConcurrentConnections = 20_000;
    options.Limits.MaxConcurrentUpgradedConnections = 500;
    options.Limits.MaxRequestBodySize = 5 * 1024 * 1024;
    options.Limits.MinRequestBodyDataRate =
        new MinDataRate(bytesPerSecond: 240, gracePeriod: TimeSpan.FromSeconds(10));
    options.Limits.RequestHeadersTimeout = TimeSpan.FromSeconds(15);
});

What interviewers look for: whether you treat these as capacity-derived numbers rather than defaults you copy from a blog post, and whether you understand that Kestrel limits are one layer in a chain that includes the OS socket backlog, the reverse proxy, and any API gateway rate limiting in front of it.

• Common mistakes: disabling limits entirely "to be safe," which removes the process's only defense against resource exhaustion; tuning Kestrel while ignoring identical limits enforced by nginx, YARP or an Azure Application Gateway sitting in front of it.
• Follow-up questions: how would you validate these limits under real load? What is the difference between rejecting a connection at the TCP layer versus returning a 503 at the application layer?

Q2 Walk through what happens, end to end, when 50,000 concurrent requests hit an async endpoint that calls a downstream database.#

Short answer: Each request gets a connection and, briefly, a thread pool thread to run the handler up to its first await. From there the thread returns to the pool, the request's state is captured in the async state machine, and the thread pool thread is only reused when the I/O completes, so the number of concurrent requests you can sustain is bounded by memory and downstream capacity, not by thread count.

The sequence matters. Kestrel accepts the TCP connection and, once request headers have been parsed, schedules the request onto the ASP.NET Core pipeline, which runs on a thread pool thread. If the handler is written with async/await all the way down, the thread executes synchronous code (routing, model binding, your own logic) until it hits an await on a genuinely asynchronous operation such as an ADO.NET or Entity Framework Core call. At that point the thread is released back to the pool; no thread is blocked waiting for the database. The pending operation continues on OS-level async I/O (completion ports on Windows, epoll on Linux), and when the database responds, a thread pool thread — not necessarily the same one — resumes the continuation. With 50,000 requests in flight, you do not have 50,000 threads; you have a much smaller number of thread pool threads doing short bursts of CPU work interleaved with a large number of pending, essentially free, I/O operations. The limiting resources become the number of open database connections (bounded by your connection pool), the memory held by each pending request's state, and the thread pool's ability to keep up with the rate of completions, which depends on ThreadPool.SetMinThreads and the runtime's hill-climbing algorithm for growing worker threads under sustained load.

If any part of that chain calls a blocking API — .Result, .Wait(), a synchronous database driver call, or Console.WriteLine to a redirected, slow stream — a thread pool thread is held hostage for the duration, and because the pool grows workers slowly by design, a burst of blocking calls can starve the pool and cause a cascading, hard-to-diagnose latency spike across unrelated requests.

What interviewers look for: a clear mental model of "small number of threads, large number of pending operations," not "one thread per request." Strong candidates mention connection pool sizing and thread pool starvation unprompted, since both are the most common real-world causes of throughput collapse under load, alongside a correct explanation of why async/await doesn't add threads, it removes the need for them.

• Common mistakes: claiming async creates a new thread for the await; forgetting that a synchronous call anywhere in the chain (including in a third-party library or a logging sink) can block a pool thread.
• Follow-up questions: what does ThreadPool.SetMinThreads actually change, and why is raising it a band-aid rather than a fix? How would IAsyncEnumerable change this picture for a streaming endpoint?

See Async/Await in C#: A Deep Dive for the underlying state-machine mechanics.

Q3 What problem does System.IO.Pipelines solve that Stream-based I/O doesn't, and when would you use PipeReader/PipeWriter directly?#

Short answer: System.IO.Pipelines solves the buffering and backpressure problems that make high-performance network parsing hard with Stream: it lets you read available bytes without committing to how much of them you've "consumed" versus merely "looked at," which avoids copying data into growing buffers while you wait for a complete message to arrive.

A naive Stream-based parser that reads a length-prefixed or delimiter-terminated message typically reads into a buffer, checks if a full message is present, and if not, copies what it has into a bigger buffer and reads more — an O(n²) pattern under fragmentation, and one that allocates constantly. Pipelines separates "how much data have I examined" from "how much data have I fully consumed" through PipeReader.ReadAsync, which returns a ReadResult wrapping a ReadOnlySequence<byte> — a view over one or more non-contiguous buffer segments, no copying required — and PipeReader.AdvanceTo(consumed, examined), which tells the pipe exactly how much to keep for the next read versus how much has already been parsed. PipeWriter mirrors this on the write side, and a Pipe connects a writer to a reader with configurable backpressure via PauseWriterThreshold and ResumeWriterThreshold: once the reader falls too far behind, writes start await-ing until the reader catches up, instead of memory growing without bound. Kestrel and SignalR are both built on System.IO.Pipelines internally, which is a large part of why Kestrel can parse HTTP framing with so few allocations.

async Task ReadMessagesAsync(PipeReader reader, CancellationToken ct)
{
    while (true)
    {
        ReadResult result = await reader.ReadAsync(ct);
        ReadOnlySequence<byte> buffer = result.Buffer;

        while (TryParseMessage(ref buffer, out ReadOnlySequence<byte> message))
        {
            ProcessMessage(message);
        }

        reader.AdvanceTo(buffer.Start, buffer.End);

        if (result.IsCompleted)
        {
            break;
        }
    }

    await reader.CompleteAsync();
}

Reach for PipeReader/PipeWriter directly when you're implementing a custom protocol over raw sockets or NetworkStream — a binary TCP protocol, a custom chunked format, a proxy — where you control framing and need minimal-allocation parsing. For ordinary HTTP request and response bodies, ASP.NET Core already exposes HttpRequest.BodyReader and HttpResponse.BodyWriter as pipes, so you rarely construct a Pipe by hand in application code.

What interviewers look for: understanding that Pipelines is about avoiding copies and giving the parser explicit control over buffer lifetime, not just "a faster stream." Mentioning ReadOnlySequence<byte> and the consumed/examined distinction is a strong signal of hands-on experience.

• Follow-up questions: why does AdvanceTo take two arguments instead of one? How would you size PauseWriterThreshold for a producer that is much faster than its consumer?

Q4 JSON serialization is often the hottest path in a high-throughput API. How do you measure and reduce its cost?#

Short answer: Treat serialization like any other hot path: profile before guessing, then attack allocations first (reflection metadata, intermediate strings and buffers), then attack CPU (source generation, Utf8JsonWriter), and finally attack volume (only serialize what the client actually needs).

Profile with dotnet-trace or a micro-benchmark under BenchmarkDotNet before changing anything; "JSON is slow" is frequently a symptom of something else, such as an EF Core query pulling in a huge object graph that then gets serialized in full. Once you've confirmed serialization itself is the cost, the highest-leverage change is usually switching System.Text.Json from its reflection-based default to source generation: a JsonSerializerContext generated at compile time removes runtime reflection and metadata caching, reduces startup cost, and is required for Native AOT. Reuse a single, cached JsonSerializerOptions instance across requests rather than constructing one per call — construction is not free, and options instances are thread-safe for reads once configured. For request and response bodies at very high volume, serialize directly to the response body stream with Utf8JsonWriter or JsonSerializer.SerializeAsync instead of first serializing to a string and then writing that string, which doubles the allocation and adds a UTF-16-to-UTF-8 conversion you don't need. Finally, look at payload shape: over-fetching fields the client discards costs both serialization CPU and network bytes, and a narrower DTO is often a bigger win than any serializer-level optimization.

[JsonSourceGenerationOptions(PropertyNamingPolicy = JsonKnownNamingPolicy.CamelCase)]
[JsonSerializable(typeof(OrderResponse))]
[JsonSerializable(typeof(List<OrderResponse>))]
internal partial class AppJsonContext : JsonSerializerContext;

app.MapGet("/orders/{id:guid}", async (Guid id, IOrderStore store) =>
{
    OrderResponse? order = await store.FindAsync(id);
    return order is null ? Results.NotFound() : Results.Json(order, AppJsonContext.Default.OrderResponse);
});

What interviewers look for: a measurement-first mindset, and awareness that source generation is primarily a CPU and startup win, not a magic fix for a badly shaped payload. Bonus points for mentioning that System.Text.Json supports both metadata-based and serialization-only ("fast path") source generation modes, and that the fast path can't be used for asynchronous streaming serialization.

• Common mistakes: benchmarking serializers on tiny payloads in isolation and extrapolating to production traffic patterns that are dominated by network and database time instead.
• Follow-up questions: when would Newtonsoft.Json still be the right call despite the overhead? How would streaming a large array with JsonSerializer.DeserializeAsyncEnumerable change memory behavior compared to deserializing the whole payload up front?

See JSON Serialization Performance Interview Questions for a deeper set of questions on this exact topic.

Q5 How do you use pooling safely in a high-throughput service, and what goes wrong when it's done carelessly?#

Short answer: Pooling trades allocation cost for a small amount of bookkeeping and discipline: you must always return what you rent, treat rented buffers as containing garbage until you write into them, and never let a pooled object escape into a cache, a field, or a background task after it's returned.

ArrayPool<T>.Shared is the workhorse for byte and char buffers in hot paths — reading a request body into a buffer, building an intermediate representation before serialization, or chunking a large response. The contract is strict: Rent may give you an array larger than requested, so track the length you asked for separately; the returned array's contents are not cleared unless you pass clearArray: true to Return, so code that relies on zeroed memory must clear it explicitly; and calling Return twice, or using the array after returning it, corrupts state for the next renter without throwing — it's a silent bug, not a loud one. Microsoft.Extensions.ObjectPool generalizes the same idea to arbitrary reusable objects, such as StringBuilder instances or reusable parser/formatter state, via ObjectPool<T> and a pluggable IPooledObjectPolicy<T> that controls how objects are reset before reuse.

byte[] buffer = ArrayPool<byte>.Shared.Rent(minimumLength: 4096);
try
{
    int read = await stream.ReadAsync(buffer.AsMemory(0, 4096), ct);
    ProcessChunk(buffer.AsSpan(0, read));
}
finally
{
    ArrayPool<byte>.Shared.Return(buffer, clearArray: true);
}

Pooling goes wrong in three recurring ways. First, retaining a reference after returning — a background task captures the buffer in a closure and reads it after another caller has already rented and overwritten it, producing intermittent, load-dependent data corruption that's brutal to reproduce. Second, pooling objects with mutable internal state that isn't fully reset, so state leaks between logically unrelated requests — a classic source of "request A saw request B's data" bugs. Third, over-pooling: wrapping every small, short-lived allocation in pooling machinery adds complexity and lock/interlocked overhead that can lose to the generation-0 garbage collector, which is already extremely cheap for small, short-lived objects. Pooling earns its keep for large or frequently reused objects on genuinely hot paths, not everywhere.

What interviewers look for: respect for the sharp edges — uncleared memory, double-return, and lifetime escaping — rather than treating pooling as a free performance switch. Experienced candidates volunteer the try/finally pattern and mention that pooling is a targeted optimization, not a default.

• Follow-up questions: how would you detect a double-return or use-after-return bug in production? When would you choose Span<T>-based stack allocation over pooling instead?

Q6 Design backpressure and load shedding for a service that occasionally receives more traffic than it can handle.#

Short answer: Decide, explicitly, what happens when demand exceeds capacity — queue with a bound, shed the least valuable work, or slow producers down — because the default behavior of an unbounded queue is to convert an overload event into an out-of-memory crash or a latency spike that outlives the traffic burst.

Backpressure means pushing the "you're going too fast" signal back toward the source instead of absorbing it in an ever-growing buffer. In ASP.NET Core, the rate limiting middleware (Microsoft.AspNetCore.RateLimiting) gives you fixed-window, sliding-window, token-bucket and concurrency-limiter algorithms you can apply per endpoint, rejecting excess requests with a 429 and, ideally, a Retry-After header so well-behaved clients back off instead of retrying immediately and making things worse. A concurrency limiter — a semaphore-based gate in front of an expensive resource — is often more useful than a raw rate limit, because it caps in-flight work rather than requests per second, which is what actually protects a downstream dependency with a fixed connection pool. Kestrel's MaxConcurrentConnections and the ASP.NET Core queue length are the coarser, connection-level version of the same idea.

Load shedding is the decision to actively drop work rather than let it queue at all once you're over capacity: reject new work immediately with a fast 503 instead of accepting it and making it wait behind everything already queued, which is almost always the better choice once queue wait time would already exceed the client's timeout — a request that will time out anyway should never be started. Prioritization matters here: health checks, authentication, and paying customers' requests should be shed last, and a useful pattern is to tag requests with a priority and shed low-priority background or batch work first, keeping user-facing latency stable during a spike. System.Threading.Channels with a bounded capacity and BoundedChannelFullMode.DropOldest or a custom rejection policy is a common building block for implementing bounded queues with explicit shedding behavior inside a service, as opposed to at its edge.

What interviewers look for: a clear distinction between backpressure (slow the producer) and load shedding (drop the work), and the judgment to know that an unbounded, "just queue it" default is a production incident waiting to happen. Mentioning that a request already past its useful deadline should be shed rather than processed is a strong, practical signal.

• Common mistakes: adding rate limiting only at the API gateway and assuming internal services never need it; using an unbounded Channel<T> or ConcurrentQueue<T> as an implicit, invisible queue with no shedding policy at all.
• Follow-up questions: how does load shedding interact with client-side retries, and how could naive retries turn a partial outage into a full one? What's the difference between shedding at the edge versus inside the service?

See ASP.NET Core Performance and Scalability Interview Questions for more on scaling patterns beyond a single service instance.

Q7 How does Server GC, including DATAS, affect throughput and latency compared to Workstation GC, and how do you tune it?#

Short answer: Server GC uses one heap and one dedicated GC thread per core and favors throughput at the cost of memory footprint; Workstation GC uses a single heap and favors a small footprint and lower pause impact on the calling thread. DATAS narrows that trade-off by letting Server GC adapt the number of active heaps to the application's actual working set instead of always using every core's heap at full size.

Server GC is the default for ASP.NET Core apps and is almost always the right choice for a high-throughput service: collections happen in parallel across heaps, and because each heap is sized around a share of the workload, gen-0 collections stay cheap even under heavy allocation rates from many concurrent requests. The cost is memory: with one heap per core, a machine with 32 cores can commit substantially more memory for the GC than a Workstation-GC process would, which is wasteful for services that don't have enough concurrent allocation to use all of it — a common situation for a service that's mostly waiting on I/O between bursts of allocation. Dynamic adaptation to application sizes (DATAS) addresses exactly this: it adapts the number of heaps and the gen-0 budget so that heap size grows roughly proportionally with the application's long-lived data size, instead of assuming maximum concurrency from process start. DATAS has been enabled by default since .NET 9; you can tune its aggressiveness with the target throughput cost percentage setting (DOTNET_GCDTargetTCP, default 2%) if a workload needs the trade-off shifted further toward throughput or further toward memory. Concurrent (background) GC, on by default alongside Server GC, further reduces the pause impact of full gen-2 collections by doing most of the marking work on a background thread while the application keeps running.

<PropertyGroup>
  <ServerGarbageCollection>true</ServerGarbageCollection>
  <ConcurrentGarbageCollection>true</ConcurrentGarbageCollection>
</PropertyGroup>

Tune further only after profiling shows GC as the bottleneck: GCHeapHardLimit caps total GC memory in a container so the GC doesn't assume it owns the whole host, GCHeapCount caps the number of heaps directly, and GCConserveMemory trades some throughput for a smaller footprint on a sliding 0-to-9 scale. In practice, most high-throughput services should start with Server GC and DATAS at their defaults and only override them once dotnet-counters shows gen-2 collection frequency or pause time actually hurting the latency SLO.

What interviewers look for: the ability to name the actual trade-off (throughput and parallelism versus memory footprint) rather than a rule of thumb like "server GC is always better," and awareness that DATAS changed the default calculus by making Server GC adapt instead of always maximizing heap count.

• Follow-up questions: how would running in a container with a CPU limit but no memory limit confuse the GC's heap count decision? What's the difference between a gen-0, gen-1 and full blocking gen-2 collection in terms of what each pauses?

See Garbage Collection in .NET: Generations, Modes and Tuning for the underlying generational model.

Q8 You're asked to increase throughput 3x on an existing service. Walk through your process, and explain the fundamental tension between throughput and latency.#

Short answer: Measure first to find the actual bottleneck — CPU, a downstream dependency, lock contention, or allocation pressure — because tuning the wrong layer wastes effort and can make things worse; then apply the fix with the best throughput-per-unit-of-added-latency ratio, since almost every throughput technique borrows from latency to get there.

Start with a baseline: current requests-per-second at your latency SLO, not peak RPS at any latency, because a service that technically serves more requests while breaching its p99 target hasn't actually solved the problem. Profile under representative load with dotnet-trace and dotnet-counters to find where time actually goes — CPU-bound work, thread pool queue length, GC pause time, lock contention, or time blocked on a downstream call. Only then choose a lever: horizontal scaling adds capacity without changing per-request latency, provided the workload is stateless and downstream dependencies (particularly a shared database) can absorb the extra connections; batching downstream calls increases throughput per connection but adds queueing delay to whichever request has to wait for the batch to fill; caching removes repeated work entirely, which is the rare case that improves both metrics at once; and increasing concurrency limits raises the throughput ceiling only if the bottleneck resource actually has spare capacity, otherwise it just moves contention downstream and can make tail latency worse.

The tension is structural, not incidental. Throughput is work completed per unit time; latency is time per unit of work; and any system with a queue — and every real system has one, even if it's just the thread pool's work queue — links them through Little's Law: the average number of requests in the system equals arrival rate multiplied by average time in the system (L = λW). Push arrival rate up without adding capacity, and either the number of requests in flight grows (more memory, worse latency) or you start rejecting work. Batching is the clearest illustration: grouping ten database writes into one round trip increases throughput per connection, but request number one in the batch now waits for requests two through ten to arrive, trading its latency for the group's efficiency. A credible 3x plan names which lever it's pulling, what it costs in latency, and what the new bottleneck will be once the old one is gone.

What interviewers look for: a repeatable process (measure, find the bottleneck, pick a lever, predict the next bottleneck) instead of a grab-bag of tips, and explicit acknowledgment that most throughput gains have a latency cost somewhere in the system, even when it isn't visible in the metric you're optimizing.

• Follow-up questions: how would you explain Little's Law to a product manager pushing for both higher throughput and lower latency simultaneously? When does adding more instances stop helping?

Q9 p50 latency looks great but p99 and p999 are terrible. How do you find and fix tail latency problems that don't show up in averages?#

Short answer: Averages and even the median hide tail latency by design, since they describe the typical request, not the request that got unlucky; you find tail latency causes by correlating slow individual traces against system-level events — GC pauses, thread pool growth, lock contention, downstream slowness — rather than by staring at an aggregate dashboard.

The first step is making the tail visible and attributable: instrument with OpenTelemetry so every request has a distributed trace, export latency as a histogram (not just an average) so you can query p99 and p999 directly, and make sure your tracing captures enough span detail — database call duration, external HTTP call duration, time spent queued before the handler even started — to see where in the request a slow one spent its time, not just that it was slow. Cross-reference slow traces against dotnet-counters or exported runtime metrics for the same time window: a cluster of slow requests lining up with a gen-2 GC pause, a thread pool thread-count increase (a sign of starvation), or a spike in lock contention counters each points to a different fix. Common, recurring causes of tail latency in .NET services include: gen-2 or blocking GC pauses on a service with a large or bursty live object graph; thread pool starvation from a hidden blocking call, which only shows up once load is high enough to exhaust the currently-warm threads; lock or database-connection-pool contention that's invisible at low concurrency and severe at high concurrency; JIT tiered compilation and warm-up effects on cold code paths hit only occasionally; and, in containerized environments, CPU throttling from a CPU limit that's stricter than it looks once multiple pods share a node.

A subtlety worth raising unprompted: load-testing tools that wait for a response before sending the next request (closed-loop generators) systematically hide tail latency, because a slow response naturally throttles the offered load — this is the "coordinated omission" problem. An open-loop generator that sends requests at a fixed rate regardless of response time gives a much more honest picture of p99 and p999 under real, bursty production traffic, where clients don't wait politely for your service to catch up.

What interviewers look for: fluency with percentiles as a first-class metric, not an afterthought, and a concrete diagnostic workflow that connects an application-level symptom (a slow trace) to a runtime-level cause (GC, thread pool, locks) using real tools, not guesswork.

• Common mistakes: optimizing for average latency and declaring victory while p99 stays flat or gets worse; using a closed-loop load generator and concluding tail latency isn't a problem because the tool never surfaced it.
• Follow-up questions: why can p999 latency matter more than p99 for a service that fans out to many downstream calls per request? How would you set an SLO that actually reflects user experience?

Q10 How do you load test and capacity-plan a high-throughput .NET service so staging numbers still hold in production?#

Short answer: Treat capacity planning as an experiment design problem: match the load generator's behavior, the payload shapes, and the dependency behavior to production as closely as possible, then push past the target load to find the actual failure mode, not just the point where you stopped testing.

Staging numbers mislead in predictable ways. A load test that reuses one warm HTTP connection undercounts the cost of TLS handshakes and connection setup that real, geographically distributed clients pay. Synthetic payloads that are smaller or more uniform than production data understate serialization and database cost. Test data that all lives in a warm cache flatters cache hit rate in a way real traffic, with its long tail of rarely-requested keys, never will. And a downstream dependency mocked with a fixed, fast response time hides the queueing behavior that a real, occasionally-slow dependency introduces under load. The fix for all of these is the same: make the load test as close to a production traffic replay as you can — real payload distributions, realistic connection churn, and either the real downstream dependency at reduced scale or a fault-injecting mock that reproduces its actual latency distribution, including its own tail.

Push the test past your target: find the load at which p99 breaches your SLO, and separately find the load at which the service falls over entirely, because "how does it fail" is as important as "how much can it take." A service that degrades gracefully — shedding low-priority work and keeping p99 bounded — is in a fundamentally better place than one that's fine until it isn't and then times out completely. Use an open-loop load generator (see the tail latency discussion above) so the test doesn't self-throttle exactly when things start going wrong, and watch the load generator itself for saturation — a test client pegged at 100% CPU is measuring the client, not your service. In production, back up synthetic testing with canarying and, where feasible, shadow traffic against a new version before it takes real load, since no staging environment perfectly reproduces the noisy-neighbor effects, network topology and data distribution of production.

What interviewers look for: awareness that most load testing failures are testing-methodology failures, not service failures, and a bias toward finding the actual breaking point rather than rubber-stamping a number that happened not to fail.

• Follow-up questions: how would you decide the right ramp-up shape for a load test — instant step, linear ramp, or a shape that mimics your real traffic pattern? What would make you trust a load test result enough to change a capacity plan based on it?

See Performance Profiling Interview Questions and Benchmarking .NET Code with BenchmarkDotNet for the tooling side of this process.