System design

#A framework for system design answers

An open design question tests trade-off decisions. Establish scope before drawing. For a store, distinguish catalog from checkout, ask about regions, and clarify whether orders may remain pending while payment is confirmed.

1. Define functional requirements, non-goals and users.
2. Ask about peak load, data size, latency SLO, availability, RPO and RTO.
3. Estimate quantities and state assumptions.
4. Define APIs, the data model and invariants.
5. Draw the simplest working path, then explore the hardest issue: consistency, fan-out or security.
6. Challenge the design with an outage and traffic spike, explaining measurement, scaling and cost.

RPO is acceptable data loss; RTO is the target recovery time. Both need business context. In a 45-minute interview, an illustrative allocation is 5 minutes clarifying, 5 estimating, 10 sketching, 20 exploring details and 5 discussing follow-ups; adapt to the interviewer.

#Back-of-envelope estimates and latency budgets

Numbers keep infrastructure decisions grounded. State units and distinguish averages from peaks and requests from business operations. The following figures are exercise assumptions, not benchmarks or performance promises.

50,000 orders/day / 86,400 seconds/day ~= 0.58 orders/second average
Assume peak = 10 * average ~= 5.8 orders/second
At 8 API calls/order: ~= 46 order-related requests/second at peak
At 2 KB/order: ~= 100 MB/day raw, before lines, indexes, replicas and backups
1 million catalog reads/day ~= 11.6 reads/second average
At 200 requests/second and 0.15 seconds average response time:
Little's law: average in-flight requests ~= 200 * 0.15 = 30
Use a stable system boundary; queue wait belongs in response time.

Measure p50, p95 and p99 in the actual deployment. End-to-end p99 is not obtained by simply adding every service's p99; identify sequential and parallel paths and measure the complete journey. Low estimated QPS does not prove capacity: verify with load tests and query inspection.

#Building blocks: each box has a cost

A stateless API does not depend on one instance's memory for session correctness. The system still has state in databases, queues and object storage, all requiring recovery plans.

#Design 1: online-store order processing

Assume one region, modest order volume, no overselling and no duplicate charges. Start with a modular monolith and a relational database. Catalog reads can tolerate staleness, but reserving the final item requires an atomic decision.

Client -> LB -> Stateless .NET API -> DB: orders, reservations, idempotency, outbox
                       |                                      |
                       v                                      v
                Payment provider                   Outbox relay -> broker
                       |                                      |
             Verified webhook -> API                  Notifications / shipping
                       ^
              Reconciliation worker

POST /orders                  Idempotency-Key scoped to customer and operation
GET  /orders/{id}              Ownership check
POST /payments/webhooks        Verify signature, amount, currency and event ID

1. In one transaction, reserve stock and persist the pending order, idempotency key, payload hash and outbox. A unique constraint prevents concurrent reuse; reject the same key with a different payload.
2. After commit, call the payment provider outside the database transaction using a supported idempotency key. A timeout is an unknown outcome, not a decline.
3. A verified webhook performs a conditional state transition and records its event ID; duplicate callbacks must not repeat effects.
4. Reconciliation resolves pending attempts. Coordinate reservation expiry with late payment: renew stock or refund according to explicit policy.
5. An outbox relay can publish twice after a crash between publishing and marking sent. Consumers need idempotency; ordering, where required, is per order rather than global.

-- (1) Inside the transaction that also creates the order and reservation.
UPDATE Stock
SET Available = Available - @quantity
WHERE TenantId = @tenant AND Sku = @sku
  AND Available >= @quantity;
-- (2) Require exactly one affected row; otherwise roll back and report unavailable.

External payment is outside the database's ACID boundary. Splitting ownership into services requires a state machine, compensation and monitoring, beyond moving methods behind HTTP.

#Design 2: URL shortener

Support link creation, fast redirection, expiry and abuse takedown, assuming a read-heavy workload. Use Links(Code PK, Destination, OwnerId, ExpiresAt, Disabled). Validate HTTP/HTTPS schemes, URL length and mutation ownership.

1. POST /links generates a sufficiently random Base62 code and inserts under a unique constraint, retrying collisions. A Base62 sequence is simpler but enumerable.
2. GET /{code} checks a hot-mapping cache, then the database, including expiry and disabled status. Cache TTL must not outlive expiry.
3. Use 302 for changeable destinations with deliberate cache policy; 301 is permanent and can persist in clients. Set Cache-Control explicitly rather than relying on status alone.
4. Bounded asynchronous analytics must not prevent redirection when analytics fails.

7 Base62 characters: 62^7 = 3,521,614,606,208 possible codes
Large address space does not eliminate random collisions.
Redirect: client -> edge/cache -> API -> Links DB
                                      -> async click event

A short code is not an access token; private files need separate authorization. If a preview fetches the destination server-side, protect against SSRF and redirects into internal networks. Immediate takedown needs explicit edge and cache invalidation.

#Design 3: notification delivery

The same event can arrive twice, and a provider may accept a message before timing out. Consume an outbox event, apply user preferences and consent, choose template and language, and persist the notification with channel tasks before delivery.

OrderShipped -> preferences + templates -> delivery records
                                             |
                       Email queue / SMS queue / Push queue
                                             |
                    Rate-limited workers -> provider -> status callbacks
Unique logical delivery: (EventId, RecipientId, Channel)
States: Pending -> Sending -> Accepted -> Delivered / Failed / Unknown

Distinguish accepted from delivered. Use bounded backoff and jitter for transient failures, and dead-letter handling with a controlled replay process. Use provider idempotency when available; local deduplication cannot guarantee exactly-once after a crash. Failing over while the outcome is unknown may duplicate delivery, so define an appropriate policy. Respect unsubscribe and quiet hours, and use expiring links for sensitive attachments instead of logging their contents.

#Design 4: rate limiter

Clarify whether the goal is limiting concurrent work or requests over time. Fixed windows are simple but allow boundary bursts. Sliding windows smooth boundaries; token buckets allow bounded bursts with sustained refill. A concurrency limiter caps in-flight work rather than requests per minute.

An in-memory limiter applies separately to each API instance. Use a gateway or atomic distributed state for a global quota. Partition by tenant, user and endpoint as required; IP-only keys penalize users behind shared NAT.

Token bucket state per key: tokens, lastRefillTime
Atomically:
  tokens = min(capacity, tokens + elapsed * refillRate)
  if tokens >= requestCost: subtract cost and allow
  otherwise: reject with 429 and a meaningful Retry-After when available
Never split read/refill/decrement into independent distributed commands.

Choose fail-open versus fail-closed if the counter store fails: a catalog may use a local fallback, while expensive endpoints need stricter protection. Limit key cardinality and expiry to prevent memory abuse. Prefer a short bounded queue or rejection over unbounded waiting.

Rate limiting design

#Design 5: file upload service

Buffering a large file in API memory wastes bandwidth and RAM. Separate the control plane that authenticates and authorizes uploads from the data plane that transfers bytes to object storage.

1. POST /uploads checks quota and ownership, creates a random object key and pending record, and returns a short-lived signed URL scoped to a specific operation.
2. The client uploads directly, using multipart or resumable transfer for large files. Never grant permanent storage credentials.
3. A callback or POST /uploads/{id}/complete triggers server-side verification of object existence, size, checksum and actual type; the client claim is insufficient.
4. Keep content quarantined until malware and policy checks finish. Transition idempotently to ready or rejected.
5. Downloads need authorization and expiring access. Periodic cleanup removes incomplete uploads and orphaned objects.

Client -> API: authorize, reserve quota, issue upload permission
Client -> private object storage: upload bytes
Storage event -> queue -> scanner -> metadata DB: Ready / Rejected
Client -> API: authorize download -> expiring object URL

The database and object store do not share one transaction. Use a state machine and reconciliation, binding approval to the exact scanned object version so content cannot be replaced afterward. Verify which restrictions the provider's signed URL can enforce, and validate the remainder server-side.

Restricted direct-storage access

#Multi-tenancy and isolation choices

A tenant is not a user; one organization may contain users with different permissions. Resolve tenants from trusted identity and scope indexes, unique constraints, caches, jobs and rate limits appropriately. EF filters help, while RLS can add database defense; review raw SQL, administrative paths and pooled connection state.

Separate databases on one server do not provide complete compute isolation. Clarify whether isolation concerns data, resources, region or encryption keys.

Multitenant architecture guidance

#Scaling a .NET API and preserving reliability

1. Find the bottleneck with traces and load tests: CPU, database, dependency or thread pool. Define an SLO first.
2. Fix N+1 queries, indexes, pagination and async I/O. Watch connection pools and downstream capacity.
3. Cache reads that tolerate staleness and use a CDN for files; test cache outages and stampedes.
4. Run stateless instances behind a load balancer, externalizing required session state. Test readiness and graceful shutdown.
5. Queue deferrable work with bounded workers; return 202 with a status resource where the contract allows.
6. Add read replicas with a lag policy, and shard only when measured needs justify complexity.

Redundancy requires separate failure domains and tested failover. Backups need restore drills. Graceful degradation may disable recommendations while preserving checkout; backpressure reduces admission when consumers fall behind. Monitor queue age as well as length, and use deadlines, retry budgets and circuit breakers for dependencies.