CitedWell

OpenAI's errors are 75% quota exhaustion. Gemini's are 95% server overload.

Two earlier posts established that provider API errors happen often, do not corrupt a score, and vary sharply by category. Neither one looked at what kind of error each engine actually throws. An "error rate" that lumps together a quota limit, a transient server outage, and a runner deciding not to call an engine at all is really three different problems wearing one label. We pulled the raw error field from every response, including errored ones, across the same 270 live audit panels and classified each failure by type.

Same word, three different failures

Every errored response carries the provider's raw error message (or, for one failure mode, the runner's own message). We classified each one into rate-limit/quota (HTTP 429 or an explicit quota message), server-side (HTTP 5xx or an explicit overload message), or runner-side cascade-skip (our own circuit breaker declining to call an engine again after an earlier quota or auth failure in the same run), plus a small leftover bucket for network-level failures (a failed fetch or a client-side timeout) that are not a provider response at all.

EngineTotal errorsRate-limit/quotaServer-side (5xx)Cascade-skipNetwork/other
OpenAI3,9252,940 (74.9%)0 (0.0%)983 (25.0%)2 (0.1%)
Perplexity1,213863 (71.1%)0 (0.0%)0 (0.0%)350 (28.9%)
Gemini4110 (0.0%)391 (95.1%)0 (0.0%)20 (4.9%)
Claude0n/an/an/an/a

These totals match the per-engine error rates from the category breakdown post exactly when blended back to one number: OpenAI 3,925 of 5,225 responses (75.1%), Gemini 411 of 2,737 (15.0%), Perplexity 1,213 of 3,571 (34.0%), Claude 0 of 2,680 (0.0%). The type breakdown is new; the underlying counts are the same dataset re-read this session.

OpenAI and Gemini fail in almost opposite ways. Nearly every OpenAI error that reaches the provider is a 429 quota message. Nearly every Gemini error is a 503 "currently experiencing high demand" message, a transient server-side condition with no quota language at all. Perplexity leans the same direction as OpenAI, mostly 429 rate limits, with a meaningful minority (28.9%) of plain network-level "fetch failed" errors that never got a response from the provider to classify. Claude logged zero errors across 2,680 real calls, so there is no failure mode to report.

A quarter of OpenAI's logged errors, 983 of 3,925, are not API failures at all. They are our own runner's circuit breaker: once a call fails on quota or auth, the runner stops sending further OpenAI calls for the rest of that run rather than repeatedly hitting the same wall. Strip those out and the actual API calls OpenAI attempted failed with a 429 quota error 2,940 times out of 2,942, essentially all of them. The "OpenAI error rate" you see quoted elsewhere is one part real quota exhaustion and one part the runner protecting the rest of the run from burning more attempts against a quota that has already run out.

Why the failure type matters more than the failure rate

A 429 and a 503 call for opposite responses. A 429 means the account is out of quota or hitting a rate ceiling; retrying the same call sooner does not help, and sending it again just spends another attempt against a limit that has not reset. A 503 means the provider's servers are temporarily overloaded; a short backoff and retry has a real chance of succeeding on the next try, because nothing about the account changed, only server load. Treating OpenAI's quota errors and Gemini's server overload as the same "error, please retry" condition means either retrying OpenAI calls that cannot succeed until quota resets, or giving up on Gemini calls that would likely have worked a few seconds later.

That is also why the runner's OpenAI-specific circuit breaker exists and Gemini does not have one: it is the correct response to a quota wall specifically, not a general error-handling policy. Applying the same skip-the-rest-of-the-run behavior to Gemini's transient 503s would throw away calls that a plain retry would likely have recovered.

See your brand's real visibility on all four engines, with every error classified and excluded from your score, not averaged over.

Get an AI Visibility Audit, $490

Methodology

Data drawn from 270 live, search-grounded audit panels (project management, customer support, and HR software brands), each run across four AI engines: ChatGPT with web search, Gemini with grounding, Perplexity Sonar, and Claude with web search. We read every line of results.jsonl for each panel this session, including errored calls, and classified each error's raw message field: HTTP 429 or a quota/rate-limit message as rate-limit/quota, HTTP 5xx or an explicit overload message as server-side, the runner's own "engine permanently disabled this run" message as cascade-skip, and anything else (a failed fetch, a client-side timeout) as network/other. Total response and error counts per engine match the blended per-engine rates published in the category breakdown post exactly, confirming this is the same dataset re-read, not a different sample. No development-rail or fixture data is included; all responses came from live engine calls. Data collected June-July 2026.