Home → Help

429 from an API gateway means two different things

What you are seeing

• 429 Too Many Requests
• {"code":"API_KEY_QUOTA_EXHAUSTED"}
• {"error":{"message":"Upstream rate limit exceeded, please retry later"}}
• A client with automatic retry that never stops retrying
• Insufficient account balance
• Concurrency limit exceeded for user, please retry later
• All available accounts are currently rate-limited. Please retry later.

Why it happens

HTTP has no status that means “you are out of money”. 402 Payment Required exists but is almost never used, so gateways overload 429 for both billing and throttling. The two are opposite in every practical way: one is permanent until you act, the other is transient and clears on its own.

This matters because most SDK retry helpers key off the status code alone. OpenAI's Python client retries 429 by default. Point it at a gateway with an empty balance and it will burn its full retry budget on a request that cannot succeed, then surface a timeout — which sends you looking for a network problem that is not there.

There is a third cause worth knowing about, because it looks like the second and is fixed differently: a per-account concurrency limit. The message names it directly — Concurrency limit exceeded for user, please retry later. This one is not about how many requests you sent over a minute but how many were in flight at the same moment. Backing off helps, but the real fix is to cap parallelism in your client; raising it is what caused the error. The full write-up of that message has measurements from a live gateway.

Confirm it is this

Make one request and look at the body rather than the status:

curl -s -o /tmp/r.json -w '%{http_code}\n' \
  'YOUR_BASE_URL/chat/completions' \
  -H 'Authorization: Bearer YOUR_KEY' \
  -H 'Content-Type: application/json' \
  -d '{"model":"YOUR_MODEL","messages":[{"role":"user","content":"hi"}],"max_tokens":1}'

cat /tmp/r.json

A code or message mentioning quota, balance or credit means billing. Anything mentioning rate, busy or upstream means throttling. If the body is empty, check for a Retry-After header — its presence points at throttling.

How to fix it

1. Branch on the body before retryingParse the JSON and treat billing 429s as fatal. Retrying them wastes your retry budget and hides the real cause behind a timeout.
2. Honour Retry-After when it is thereThrottling responses often carry it. When absent, exponential backoff starting near one second is a reasonable default; retrying immediately usually makes congestion worse.
3. Alert on balance separately from errorsAn empty balance is a business event, not an incident. Most gateways expose the balance through their own API or a dashboard — watch that number, and you will never meet this 429 in production.
4. Cap in-flight requests, not just the rateIf the message mentions concurrency, a slower retry loop will not help — the limit counts simultaneous requests. A semaphore around your client, sized to whatever the gateway allows, fixes it properly.
5. Do not raise concurrency to fix a throttling 429More parallel requests against a congested upstream produces more 429s, not more throughput. Lower concurrency and let backoff work.

Related

Last checked 2026-10-14. Written from problems diagnosed on a live OpenAI-compatible gateway, not collected from other sites.