Why a provider error does not corrupt your AI visibility score
An audit calls four provider APIs, and provider-side errors happen: rate limits, timeouts, a model reporting high demand. Across 271 real audit batches in our own run history, 218 of them (80.4%) hit at least one API error somewhere in the batch. 269 of the 271 (99.3%) still completed and produced a scored report. The other 2 halted on purpose rather than finish on bad data. Here is exactly what happens to an errored call between the moment it fails and the moment your score gets calculated.
The setup
Every audit runs the same prompt panel against four search-grounded engines: ChatGPT with web search, Gemini with grounding, Perplexity Sonar, and Claude with web search. Each of those is a live API call to a different provider, and each provider can fail a call for reasons that have nothing to do with your brand: a rate limit on the account tier, a timeout, a 503 while a model is under load. We pulled every result from 271 real, non-fixture batches run against these four engines and counted what happened.
How often a batch actually hits an error
| Outcome | Batches | Share |
|---|---|---|
| Batch had at least one API error | 218 of 271 | 80.4% |
| Batch completed and was scored anyway | 269 of 271 | 99.3% |
| Batch halted before scoring (by design) | 2 of 271 | 0.7% |
Individual call error rates varied a lot by provider in this sample. Claude returned zero errors across 2,726 calls. Gemini errored on 14.8% of calls. Perplexity errored on 33.5%. OpenAI errored on 68.6% of live calls, almost all of them 429 rate-limit responses hit while running many batches back to back in a short window, not a general reliability problem with the API. The point is not which provider looked worse in this particular stretch, rate limits shift with billing tier and time period. The point is that a call failing is routine, expected, and handled, not an emergency and not a data quality problem.
What happens to a single failed call
When a call to an engine fails, the pipeline checks what kind of failure it is. A transient error, a rate limit or a timeout, gets retried automatically with a backoff delay before it is recorded as a failure at all. A permanent error, an exhausted quota or an invalid key, is not retried. Retrying it would just burn another 30 to 60 seconds per attempt against an account problem that will not clear on its own, so the pipeline marks that engine dead for the rest of the run and fast-fails any remaining calls to it without spending more API budget.
Either way, if a call ends in error, it never becomes a data point in your score. The scoring step only builds an analysis from a response when there is no error attached to it. A failed call counts as no data collected for that one prompt-engine pair, and it is simply absent from the count, never as your brand failing to appear. The difference matters: a missed mention lowers your visibility rate, a missing data point does not touch it either direction.
Why 2 of the 271 batches did not get scored at all
Two batches in this sample never produced a score, and that was also by design. Both hit three consecutive batches where every single call errored, in both cases a run of Gemini 503 responses reporting the model under high demand. At that point the pipeline halts the run instead of continuing to spend API calls against what looks like a systemic outage rather than an isolated hiccup. Scoring a batch that only completed 16 of 40 calls would misrepresent what was actually measured, so the correct outcome is no report yet, not a lower-confidence report built on a fraction of the panel.
That threshold is concrete: a batch needs at least 6 completed calls with no error before it contributes to the loss-record and scoring pipeline at all. Below that, the run is treated as incomplete rather than scored thin.
What this means for a client reading a report
If your audit report shows a mention rate of 24% across 40 prompts, that 24% is computed from the prompts that actually returned a usable response, not from 40 minus however many happened to error that day. Provider outages, rate limits, and timeouts are a normal part of calling four different companies' APIs in the same run, and the pipeline is built assuming they will happen on close to every run, not as an exception case. What would corrupt a score is silently counting an error as a loss. That never happens here.
See how your brand actually scores across four AI engines, errors handled correctly and never counted against you.
Get an AI Visibility Audit, $490Methodology
271 real, non-fixture audit batches (270 ten-prompt teaser panels run June 15 to June 19, 2026, plus one 40-prompt full audit run June 12, 2026), all live-engine calls to ChatGPT, Gemini, Perplexity, and Claude with web search or grounding enabled. Error and completion counts pulled directly from each run's results.jsonl and progress.json this session. Per-call error rate excludes fast-failed calls to engines already marked dead earlier in the same run. No fixture or dev-engine data included.