Skip to main content

Distributed rate limiter (medium)

You are interviewing for Engineering Manager. This is a 45–60 minute whiteboard / word-problem. Talk through observations as you go. Draw.

This prompt is adapted from System Design School: Design a Distributed Rate Limiter (SDS labels it medium). Read this document only. Do not open the interviewer page. Do not open the SDS solution until after the mock.

Context​

You have several application servers behind a load balancer. Clients call your APIs. You need to allow or reject a request because the caller is over a configured rate.

A counter on a single process is not enough: two servers must not each allow a full quota.

This limiter sits in front of an API (or an internal RPC). Adjacent problems — billing, quotas as a product, DDoS scrubbing — can be named and deferred.

Problem​

Design a distributed rate limiter.

It should decide allow vs deny quickly. It should keep a defensible policy when a limiter node dies. You will need to choose what the limit is keyed on, how strict accuracy is, and what happens when the limiter itself is unavailable.

What to produce​

  1. Clarify requirements. Ask questions. Write down what you are optimizing for (accuracy vs latency vs availability).
  2. Sketch how a request is allowed or denied (including what the client sees on deny).
  3. Draw a high-level design. Walk two app servers hitting the same caller at once.
  4. Go deep on the hardest parts.
  5. Close as an engineering manager: what ships first, who owns what, what you measure, how you talk about risk — including fail-open vs fail-closed as a product decision, not only a Redis choice.