All terms
Glossary
Rate limiting
Also: Throttling
An upper bound on requests per period and caller, so that individual users cannot take a system down for everyone else.
Throttling protects against three things at once: accidental loops in someone else code, deliberate overload, and the cost both create.
Common approaches such as token bucket allow short bursts and only cap sustained throughput. What matters is that rejected requests answer clearly as such, including when a retry makes sense.
How you notice it
- Individual callers generate a multiple of normal load.
- There is a public interface.
- Cost scales with the number of calls.
Frequently asked
Which limit is the right one?
The one that does not disturb normal operation. Measure the 99th percentile of real users and set the limit well above it. A limit that hits legitimate users gets switched off after the first complaint and protects nothing thereafter.