What is a rate limit? (why the API told you to slow down)

What it actually is

A rate limit is a cap a service puts on how many requests you’re allowed to send it inside a given window of time, say 60 requests per minute. Go over it and the service doesn’t crash, and it doesn’t quietly let your request through either. It refuses, with an error, until the window resets.

Think of a bakery with one person at the till. That person can ring up maybe 20 customers an hour, no matter how many people are in the shop. If 40 people show up in the same five minutes, the ones at the back get told “come back in a bit”, not because the bread ran out, but because the till can only move so fast. A rate limit is an API’s version of that line.

Nearly every external service you connect to (Stripe, an email API, Google Maps, Anthropic’s own API) enforces one. AI APIs often split it two ways: a cap on requests per minute AND a separate cap on tokens per minute, so you can hit a wall even while sending relatively few, but very long, requests.

Why your AI just did this

When your agent wires up an external API, a payment processor, an email service, a call to an LLM, it’s writing code that will eventually hit that service’s rate limit once the app sees real traffic. A careful build adds retry logic for this: catch the 429, wait, try again, waiting a bit longer each time if it keeps failing (this pattern is called exponential backoff).

Most AI-generated code skips that part, not because the agent has never heard of rate limits, but because a quick build-and-test-once cycle never sends enough requests to trigger one. The feature works fine across your five test clicks in the browser, then breaks the day ten real users show up at once, or the day you loop-test something by mashing “generate” fifteen times in a row.

When you’ll run into it

• A 429 Too Many Requests error in your browser console or server logs, sometimes carrying a Retry-After header that names the exact wait.
• Anthropic’s API returning a 429 with a rate_limit_error type, worth telling apart from a 529, which means their servers are overloaded, nothing to do with how much you personally sent.
• Your app working perfectly alone, then failing the moment a handful of people use it at the same time, a classic sign nobody added retry handling.
• A third-party API (a payment processor, an email sender, a maps service) silently dropping requests mid-burst, during a bulk import or a loop firing calls too fast.

What to check

• Does the code catch a 429 response specifically, or does it lump every failed request into one generic error handler
• Is there a retry with backoff (wait, then wait longer) instead of hammering the same endpoint again immediately
• Does it read and respect the Retry-After header when the service sends one, instead of guessing how long to wait
• Are you also rate-limiting your OWN app’s endpoints, not just relying on the outside services you call, that protects you from abuse, not only from failure
• For an AI API specifically, check whether you’re hitting a requests-per-minute cap or a tokens-per-minute cap, they’re separate limits and the fix isn’t the same

Say it like a dev

Instead of: “the API is broken, it just stopped working” Say: “we’re hitting a 429, is there retry logic with backoff on this call?”

Instead of: “make it never fail, ever” Say: “add exponential backoff so it waits and retries instead of giving up on the first rate limit hit”

Instead of: “we ran out of something” Say: “is this a rate limit, too fast, or a usage cap, too much, because the fix is different for each”