OpenAI errors out 88% of the time in one category. 0.3% in another.
An earlier post established that provider API errors are common and do not corrupt an audit score, and gave a single error rate per engine: Claude near zero, Gemini and Perplexity moderate, OpenAI the highest. What that post did not check is whether those rates hold steady across the categories we audit, or whether "OpenAI's error rate" is really three different numbers wearing one label. We went back into the same result logs and split error rate by category. It is three different numbers.
The same engine, three very different failure rates
We pulled every response line, including errored ones, from the 270 live audit panels across our three tracked categories, project management, HR, and customer support software, and grouped them by category and by engine. The error field is populated straight from the provider's response, so this is exactly what each engine's API returned during a real grounded call, not a simulation.
| Engine | Project management | HR software | Customer support |
|---|---|---|---|
| OpenAI | 83.2% (2,819/3,389) | 88.0% (1,104/1,254) | 0.3% (2/582) |
| Gemini | 7.2% (124/1,730) | 36.6% (154/421) | 22.7% (133/586) |
| Perplexity | 2.7% (47/1,730) | 20.1% (86/427) | 76.4% (1,080/1,414) |
| Claude | 0.0% (0/1,730) | 0.0% (0/380) | 0.0% (0/570) |
OpenAI's error rate swings from 0.3% to 88.0% depending on the category alone. Perplexity swings from 2.7% to 76.4%, and it is almost the mirror image of OpenAI: cleanest where OpenAI is worst, worst where OpenAI is cleanest. Gemini moves in a narrower band, 7.2% to 36.6%, but still more than a five-fold spread. Claude is the one flat line, zero errors in all three categories across 2,680 calls, matching the near-zero rate the earlier post documented.
Why this is not just OpenAI running out of quota in general
If OpenAI's failures were simple account-wide quota exhaustion, the rate would be roughly flat across whatever categories happened to run that session, since quota does not know what category a prompt belongs to. It is not flat, and the sharpest evidence is the OpenAI/Perplexity mirror on customer support software specifically: OpenAI's best category (0.3% error) is Perplexity's worst (76.4% error) in that same category, on the same 582 to 1,414 calls. That pattern is consistent with per-session batch timing, which category's panels happened to run during a given provider's rate-limit window, rather than a fixed property of the category itself. We cannot fully separate "this category is harder for this engine to retrieve" from "this category's panels happened to run when the provider was rate-limiting us" with a single session's data, and we are not going to round that uncertainty away.
What we can say with confidence: the single per-engine error rate quoted in the earlier post is a blend, and the blend hides a category-level spread wide enough to change what "normal" looks like for a specific audit. If your customer support software audit shows OpenAI succeeding on nearly every call and your HR software audit shows it failing on most, that difference is not a sign something is broken with your account or your panel. It is the pattern in the data across all 270 panels we track.
Claude is the one engine where category does not matter
Across 2,680 real Claude calls spanning all three categories, zero errored. That consistency, not just the low average, is what makes Claude the dependable baseline in a multi-engine audit: whatever timing or rate-limit conditions moved OpenAI, Gemini, and Perplexity's numbers around by category, Claude's did not move at all.
See your brand's real visibility on all four engines, with every error logged and excluded from your score, not averaged over.
Get an AI Visibility Audit, $490Methodology
Data drawn from 270 live, search-grounded audit panels (project management, customer support, and HR software brands), each run across four AI engines: ChatGPT with web search, Gemini with grounding, Perplexity Sonar, and Claude with web search. We read every line of results.jsonl for each panel this session, including errored calls, and grouped by the panel's category (from panel.json) and the engine field. A response counts as errored if the provider's API returned a non-empty error field; some engines log a retry attempt as its own line, so response counts differ from the panel's prompt count. Non-error response totals per category (5,589 project management, 1,138 HR, 1,937 customer support) match the counts used in earlier posts on this same dataset. No development-rail or fixture data is included; all responses came from live engine calls. Data collected June-July 2026.