Polyglot persistence — picking a different data store for each workload instead of forcing every problem into one relational schema — is now the default assumption in distributed .NET systems, and it is squarely an architect-level topic. Interviewers use it to check whether you reason from access patterns, consistency requirements and operational cost before reaching for a new database, rather than reciting a list of products. Because a bad partition key or the wrong consistency level in a system like Azure Cosmos DB is expensive and slow to unwind, this area rewards candidates who can show their work: why a store was chosen, what it cost to operate, and what broke when the design was wrong. These ten questions cover data store selection, Cosmos DB partitioning and request units, consistency trade-offs, document modeling, change feed architecture, and the operational reality of running several databases at once.
Q1 How do you decide which data store to use for a given workload in a polyglot persistence architecture?#
Short answer: Start from the workload's access pattern, consistency requirement and growth shape, not from a preference for a technology. A relational database stays the default until a specific pattern — massive write throughput, schema-flexible aggregates at global scale, graph traversal, full-text or vector search, or high-cardinality time series — justifies something else, and every store you add is a permanent operational commitment, not a free capability.
A useful framework is to answer four questions per workload before naming a technology: what shape is the data (rows, documents, graphs, key-value pairs, vectors, time series), what are the read and write patterns (point lookups, range scans, ad hoc joins, full-text search), what consistency and transactional guarantees does the domain require, and who on the team will operate this store at 3 a.m. when it misbehaves.
| Workload | Typical choice | Why |
|---|---|---|
| Transactional core domain | Relational (SQL Server, PostgreSQL) | ACID, mature tooling, joins |
| Schema-flexible aggregates at scale | Cosmos DB or MongoDB | Horizontal scale, aggregate-shaped reads |
| Session state, cache, leaderboards | Redis | Sub-millisecond in-memory access |
| Full-text or vector search | Azure AI Search, Elasticsearch | Purpose-built ranking and indexing |
| Event log, integration backbone | Kafka or Azure Event Hubs | Ordered, replayable stream |
| High-volume telemetry | A time-series store | Compression and time-bucketed queries |
In a service-oriented or microservices architecture, let each store map to one bounded context's data, never to shared data used by several services, and see the NoSQL in .NET guide for the mechanics once you have chosen a document store.
What interviewers look for: reasoning from requirements to technology instead of a product list; explicit acknowledgment of operational cost as a decision input; a connection to bounded contexts rather than "one database per microservice" as a slogan.
Common mistakes: picking NoSQL because it is fashionable, with no concrete access pattern to justify it; ignoring whether the team can actually run and monitor a second or third database technology; forgetting that a relational database with a jsonb or json column already covers many "we need flexible schema" requests.
Follow-up questions:
- How would you introduce a new data store into an existing service without a full rewrite?
- What would make you reverse a polyglot persistence decision after it shipped?
Q2 Walk through how you would design a partition key for a Cosmos DB container serving a multitenant SaaS application.#
Short answer: Pick a key with high cardinality, even load and presence in your hottest queries, since the partition key decides both scale-out and the scope of atomic transactions. For a multitenant system, start with the tenant id; when a handful of tenants dominate volume, move to a hierarchical partition key so a large tenant's data still spreads across physical partitions while tenant-scoped queries stay single-partition.
The trap is a key that looks reasonable but concentrates load: creation date puts every write from today on one logical partition, and a status field has too few distinct values to spread anything. tenantId alone works until one tenant's data exceeds the 20 GB logical partition ceiling or dominates the container's throughput; hierarchical partition keys, available with up to three levels, solve that by composing tenantId, then a second discriminator such as userId or entityType.
var properties = new ContainerProperties(
id: "audit-events",
partitionKeyPaths: ["/tenantId", "/entityType", "/entityId"]);
// A query that supplies only the tenant prefix still avoids a full fan-out
var prefixKey = new PartitionKeyBuilder().Add(tenantId).Build();
var iterator = container.GetItemQueryIterator<AuditEvent>(
new QueryDefinition("SELECT * FROM c WHERE c.entityType = @type")
.WithParameter("@type", "Invoice"),
requestOptions: new QueryRequestOptions { PartitionKey = prefixKey });Also weigh transaction scope: atomic multi-item writes in Cosmos DB are limited to one logical partition, so if two entities must change together, they need to share a partition key. Because the partition key cannot be changed after data is written, load-test the chosen key against realistic, skewed tenant sizes before the container goes live, not after.
What interviewers look for: a concrete cardinality-and-load argument, not just "use the tenant id"; awareness that the key also bounds transactions; recognition that the decision is effectively permanent.
Common mistakes: choosing a low-cardinality or time-based key; forgetting that queries without the partition key fan out to every physical partition; assuming the key can be changed later without a data migration.
Q3 What are Request Units in Azure Cosmos DB, and how do you estimate and control RU costs?#
Short answer: A Request Unit (RU) is Cosmos DB's normalized measure of the CPU, memory and I/O an operation consumes, so a small point read by id and partition key costs about 1 RU while writes, large documents, more indexed properties and complex queries cost more. You control spend by minimizing per-operation cost, trimming the indexing policy, choosing the right throughput mode, and measuring, not guessing.
Three throughput modes trade cost predictability for elasticity: provisioned throughput reserves RU/s for a container or database, autoscale throughput scales between a floor and a ceiling automatically, and serverless bills per request with no reservation, which suits spiky or intermittent workloads. Whichever mode you pick, when demand exceeds the budget the service returns HTTP 429 and the SDK retries transparently for a bounded time before surfacing the failure, so sustained 429s are a capacity signal, not just an error to swallow.
var response = await container.ReadItemAsync<Order>(
orderId, new PartitionKey(customerId), cancellationToken: ct);
logger.LogInformation("Point read cost {Charge} RU", response.RequestCharge);Estimating cost means running representative operations against realistic document shapes, logging RequestCharge for each one, and multiplying by expected volume with headroom for peak traffic — not reading a generic "RUs per operation" table, because size, indexing and query shape all move the number. The single cheapest lever most teams miss is excluding paths you never filter or sort on from the indexing policy, since every indexed property adds write cost across the container's lifetime.
What interviewers look for: understanding that RU cost is workload-specific and must be measured; knowledge of the three throughput modes and when each fits; treating 429 as a design signal rather than a bug to retry away.
Common mistakes: provisioning throughput once and never revisiting it as data and query patterns change; leaving the default indexing policy on containers with large, rarely-queried properties; assuming cross-partition queries cost the same as single-partition ones.
Q4 Explain Cosmos DB's consistency levels and when you would choose something other than the default.#
Short answer: Cosmos DB offers five consistency levels — strong, bounded staleness, session, consistent prefix and eventual — set on the account and relaxable per request, trading recency guarantees for latency and RU cost. Session is the default and the right choice for the large majority of applications, because it guarantees a client's own writes are visible to its own subsequent reads without paying for a global quorum on every request.
| Level | Guarantee | RU/latency cost | When to choose it |
|---|---|---|---|
| Strong | Always reads the latest committed write | Highest | Single-region, strict correctness |
| Bounded staleness | Lag bounded by time or version count | High | Multi-region reads needing a known bound |
| Session | Read-your-writes within a client session | Default, low | Nearly everything user-facing |
| Consistent prefix | Writes never appear out of order | Low | Feeds, activity streams |
| Eventual | No ordering guarantee | Lowest | Counters, telemetry, non-critical reads |
Choosing strong or bounded staleness is a deliberate trade: both use quorum reads across replicas, which costs more RUs and adds latency, and strong consistency also constrains how many regions can accept writes. Eventual and consistent prefix suit read paths that tolerate staleness, such as a public activity feed, where the RU and latency savings compound at scale. This is the same trade-off that CAP theorem describes in the abstract; see CAP, consistency models and idempotency for the general treatment, and note that the account-level default can still be relaxed to a weaker level on an individual request when that specific read does not need it.
What interviewers look for: naming session as the practical default rather than always reaching for strong; connecting the choice to concrete RU and latency cost, not just a definition; awareness that consistency is configurable per request, not only per account.
Common mistakes: defaulting to strong consistency "to be safe" without pricing the cost; confusing Cosmos DB's tunable levels with the binary strong-versus-eventual framing many engineers learn first.
Q5 How do you approach document modeling — when do you embed data versus reference it?#
Short answer: Model from queries, not from entities: embed data that is read together, owned by one parent, bounded in size and changed together, and reference data that grows without bound, is shared by many parents, or is updated on its own schedule. Get this wrong and you either pay for extra round trips on your hottest read or blow through a document size limit as an array grows unbounded.
An order with its line items and shipping address is a natural embed, because they are read and written as one unit and the line items do not grow indefinitely. Comments on a popular post, or a product referenced by thousands of historical orders, should be referenced, both because the child collection is unbounded and because embedding would duplicate a mutable entity across many parents. The practical middle ground is denormalization with clear ownership: an order line copies the product name and price at purchase time, because that is a historical fact, while the product document remains the single source of truth for current pricing.
{
"id": "order-88f21c",
"type": "order",
"customerId": "cust-5102",
"lines": [
{ "productId": "prod-9", "name": "Mechanical keyboard", "unitPrice": 89.00, "quantity": 1 }
],
"total": 89.00
}Storing several related entity types in one container behind a type discriminator, such as a customer profile alongside that customer's orders, is idiomatic when they share a partition key, because it lets you read a customer's full context in one round trip. When a copied value must eventually catch up with the source of truth, propagate the change asynchronously — typically through the change feed — and accept a short window of staleness rather than a synchronous, distributed update.
What interviewers look for: a query-driven modeling process instead of a relational habit transplanted onto documents; a clear embed-versus-reference rule with size and ownership as the deciding factors; comfort with asynchronous propagation for denormalized copies.
Common mistakes: embedding an unbounded child collection, such as every comment or every event, until it hits a document size limit; treating every relationship as a candidate for embedding just because a join would have expressed it relationally.
Q6 What is the Cosmos DB change feed, and what architecture patterns does it enable?#
Short answer: The change feed is a persistent, ordered-per-partition log of every insert and update in a container, and the change feed processor distributes reading that log across running instances, checkpointing progress in a lease container. It is the backbone for read models, search indexing, cache invalidation and cross-service integration without polling or a separate message broker.
Because delivery is at-least-once and ordering is only guaranteed within a logical partition, every consumer must be idempotent: assign absolute values rather than incrementing counters, and key deduplication off the item's id and a version or timestamp when exact-once behavior matters downstream.
public sealed class OrderProjectionService(CosmosClient client) : IHostedService
{
private ChangeFeedProcessor? _processor;
public async Task StartAsync(CancellationToken ct)
{
var orders = client.GetContainer("shop", "orders");
var leases = client.GetContainer("shop", "leases");
_processor = orders
.GetChangeFeedProcessorBuilder<Order>("order-projections", HandleChangesAsync)
.WithInstanceName(Environment.MachineName)
.WithLeaseContainer(leases)
.Build();
await _processor.StartAsync();
}
public Task StopAsync(CancellationToken ct) => _processor?.StopAsync() ?? Task.CompletedTask;
private async Task HandleChangesAsync(
ChangeFeedProcessorContext context, IReadOnlyCollection<Order> changes, CancellationToken ct)
{
foreach (var order in changes)
{
await readModels.UpsertOrderSummaryAsync(order, ct); // idempotent by design
}
}
}Common patterns built on the change feed include materializing a denormalized read model for a CQRS query side, feeding a search index or cache, driving the transactional outbox pattern by writing a business document and an event in the same transactional batch and letting the processor publish it, and replicating a subset of data into another store as part of a polyglot architecture. The default mode delivers only the latest version of each changed item; an all-versions-and-deletes mode is available for consumers that also need intermediate versions and deletions.
What interviewers look for: understanding that the change feed is ordered per partition, not globally; designing consumers for at-least-once delivery; naming concrete patterns (outbox, projections, search indexing) rather than a vague "it's for events."
Common mistakes: writing non-idempotent handlers that double-count on redelivery; assuming global ordering across partitions; forgetting that deletes are invisible to the default change feed mode.
Q7 What are the operational costs of running polyglot persistence in production?#
Short answer: Every additional data store technology adds its own backup and restore story, monitoring and alerting, capacity planning, security model, upgrade cadence and on-call expertise — costs that do not show up in a proof of concept but dominate total cost of ownership within a year or two. Polyglot persistence is a legitimate architecture, not a free one, and the bill is paid in operational surface area, not licensing.
Concretely: each store needs its own disaster recovery plan and tested restore procedure; each needs dashboards and alert thresholds tuned to its own failure modes (RU throttling looks nothing like connection pool exhaustion, which looks nothing like replica lag); each needs engineers who understand its consistency model well enough to debug a 2 a.m. incident; and keeping data consistent across stores usually means solving the dual-write problem with an outbox or change feed rather than a distributed transaction, which is itself ongoing engineering work, not a one-time cost.
The pragmatic response is to bound the blast radius: let each bounded context own its data and its store choice, keep the number of distinct store technologies small even if you have many databases, invest in one team-wide set of conventions for outbox and idempotent consumers, and treat "can we operate this" as a first-class criterion before adopting a new store, not an afterthought once the design is picked.
What interviewers look for: a real answer about operations — backups, monitoring, on-call, upgrades — not just "it's more complex"; recognition of the dual-write problem and how outbox or change feed patterns address it; a bias toward fewer store technologies, chosen deliberately.
Common mistakes: treating polyglot persistence as purely a technical design exercise, ignoring staffing and on-call reality; solving cross-store consistency with a distributed transaction instead of an outbox or eventual-consistency pattern; adding a new store per team without a shared operational playbook.
Q8 How do you handle transactions across multiple documents or aggregates in a store that only supports single-partition transactions?#
Short answer: Design so that the operations requiring atomicity share a partition, using Cosmos DB's transactional batch for up to 100 operations on one logical partition; for anything crossing partitions or aggregates, replace the distributed transaction with a saga, an idempotent compensating action, or the transactional outbox pattern driven by the change feed.
A transactional batch is the right tool when a business operation is genuinely local: writing an order and an outbox message that shares the order's partition key, or updating a counter alongside the row that triggered it.
var pk = new PartitionKey(order.CustomerId);
using var result = await orders.CreateTransactionalBatch(pk)
.CreateItem(order)
.CreateItem(new OutboxMessage(Guid.NewGuid().ToString(), order.CustomerId, "OrderPlaced", order.Id))
.ExecuteAsync(ct);
if (!result.IsSuccessStatusCode)
{
throw new InvalidOperationException($"Order batch failed with {result.StatusCode}."); // all-or-nothing
}When the operation spans partitions or aggregates — reserving inventory in one container while creating an order in another — there is no ACID transaction to reach for. Model it as a saga: each step is local and reversible, a coordinator (often driven by change feed events) tracks progress, and a failure triggers compensating actions rather than a rollback. This is more code than a database transaction, but it is also what makes the system scale past a single node, and it is the same shape used for cross-service consistency in any distributed system; see distributed transactions and the saga pattern for the broader pattern.
What interviewers look for: knowing the actual scope limit of Cosmos DB transactions; reaching for sagas and outbox rather than proposing a two-phase commit; comfort designing compensating actions.
Common mistakes: assuming transactional batch works across partitions; trying to bolt a distributed-transaction coordinator onto a NoSQL store instead of redesigning around eventual consistency; forgetting that saga steps must themselves be idempotent, since retries and redelivery are normal.
Q9 How would you migrate a hot entity to a new partition key design without downtime?#
Short answer: Because a Cosmos DB partition key is immutable once data exists, there is no in-place fix — create a new container with the better key (often a hierarchical key), dual-write to both containers, backfill history through the change feed or a bulk job, verify, then cut reads over and retire the old container. The shape is the same expand-contract discipline used for relational schema changes, applied to a partition boundary instead of a column.
Concretely: stand up the new container with hierarchical partition key paths sized for the current skew; deploy application code that writes every new item to both containers; run a change-feed-driven or bulk backfill job to copy historical items into the new shape, throttled to leave RU headroom for live traffic; compare counts and spot-check documents between the two containers; then flip reads to the new container behind a feature flag so the cutover is a configuration change, not a deployment, and can be reversed instantly if something looks wrong. Only after a quiet period do you stop dual-writing and delete the old container. The same coordination concerns — dual-write correctness, backfill idempotency, feature-flag cutover, and a rollback that does not require a redeploy — apply directly to zero-downtime database migrations in a relational store.
What interviewers look for: recognizing the partition key is immutable and reasoning to a new-container migration rather than proposing an in-place change; a real expand-contract sequence with a safe cutover and rollback; awareness that the backfill must not starve live traffic of RUs.
Common mistakes: underestimating how long a full backfill takes on a large container and blocking a release on it; cutting over reads before verifying data completeness; forgetting to keep dual-writing until the old container is fully retired, which reopens a window for lost writes.
Q10 When is polyglot persistence the wrong choice?#
Short answer: Polyglot persistence is a mistake when it is applied as a rule ("every microservice gets its own database technology") rather than a response to a specific access pattern, when the team cannot staff the operational burden of the extra stores, or when the domain's consistency needs are strict enough that splitting data across stores just recreates distributed transactions you cannot cleanly solve.
Warning signs are recognizable in retrospect: a small team running four or five different database technologies for a handful of services; frequent incidents caused by data drifting out of sync between stores with no reconciliation process; business logic that constantly needs to join or aggregate across stores in real time, which is a sign the data was split along the wrong boundary; and a "database per microservice" policy applied even to services that share the same bounded context and would be simpler as one schema in one database. In all of these cases, consolidating onto fewer stores — often back to a single well-run relational database, optionally with a cache in front — reduces both bugs and on-call load more than any amount of tuning the split architecture would.
What interviewers look for: willingness to argue against polyglot persistence, not just for it; recognizing "database per service" as a guideline that can be over-applied; connecting store boundaries to bounded contexts rather than deployment units.
Common mistakes: treating more data stores as inherently more scalable or more modern; not having a concrete threshold for when to consolidate back down.
Quick-Fire Round#
| Question | Answer |
|---|---|
| Default Cosmos DB consistency level? | Session |
| Approximate RU cost of a Cosmos DB point read on a small item? | About 1 RU |
| Maximum size of a Cosmos DB logical partition? | 20 GB |
| How many operations can a Cosmos DB transactional batch contain? | Up to 100, one logical partition |
| HTTP status Cosmos DB returns when RU budget is exceeded? | 429 |
| MongoDB's equivalent scale-out key to a Cosmos DB partition key? | Shard key |
| Does MongoDB support multi-document ACID transactions? | Yes, on replica sets and sharded clusters |
| What guarantees does "consistent prefix" give? | Writes are never seen out of order |
| What pattern replaces a distributed transaction across aggregates? | Saga with compensating actions |
| What log powers Cosmos DB read models and integration? | The change feed |
How to Prepare#
- Build a small Cosmos DB container, log
RequestChargefor point reads, queries and writes, and watch the numbers move as you change the indexing policy. - Practice explaining the five consistency levels in terms of a concrete scenario (a shopping cart, an activity feed) instead of reciting the names.
- Sketch a change feed processor and an outbox pattern from memory; both come up constantly at this level.
- Prepare one generic story about a partition key or store choice that had to be redesigned, and be ready to explain the trade-off you underestimated the first time.
- Practice arguing against polyglot persistence for a scenario, not just for it — interviewers probe judgment, not enthusiasm for NoSQL.