Rate limiting per tenant, not just globally, prevents noisy neighbors

A global rate limit lets one high volume tenant starve everyone else. Per tenant limits cost more to implement and are the only version that is actually fair.

When engineering teams first expose a multi-tenant API, they almost always implement global rate limiting at the gateway level. They calculate that the backend database can safely process ten thousand concurrent queries per second. They configure the load balancer to drop any traffic exceeding that threshold. The infrastructure is now protected from being overwhelmed.

But protecting the infrastructure is not the same thing as protecting the user experience.

The Noisy Neighbor Problem In a B2B enterprise environment, tenant workloads are rarely uniform. You might have one hundred small clinics making a dozen API calls an hour, and one massive hospital network that suddenly decides to run an unoptimized bulk historical data extraction script at noon.

The hospital network instantly consumes nine thousand nine hundred of your available ten thousand requests. The global rate limiter sees the traffic remaining just under the ceiling and allows it. The database remains stable. But your hundred smaller clients are suddenly receiving HTTP 429 Too Many Requests errors, despite operating well within their expected usage.

You have allowed one noisy neighbor to effectively take down the platform for everyone else.

Pushing Identity to the Edge Fixing this requires pushing tenant awareness out to the absolute edge of your architecture. The API gateway can no longer just blindly count incoming packets. It must parse the incoming authorization token, extract the tenant identifier, and evaluate the request against a specific, isolated quota.

If the large hospital network exceeds its allocated slice of the compute pie, its requests are throttled immediately. The remaining capacity stays fully reserved for the rest of the ecosystem.

The Architectural Cost This shift introduces real infrastructure complexity. You are adding network latency to every single API request to perform the tenant lookup. You have to manage distributed memory stores like Redis to keep request counters synchronized across multiple regional load balancers. You also have to build administrative tooling to define and manage different tier limits for different customer contracts.

It is significantly harder to build and maintain than a simple global traffic throttle. But in a multi-tenant system, fairness is a core engineering requirement. A global rate limit is just a blunt instrument to prevent servers from melting. A per-tenant rate limit is a guarantee that every client receives exactly the capacity they expect, regardless of what anyone else is doing on the platform.