I expected a second AI from another company to catch more... It caught less.

A while back a question came up in here: is a second AI from a different company a better second set of eyes than asking the same AI again? (https://www.skool.com/cliefnotes/second-eyes-does-a-second-ai-actually-catch-more-im-testing-it-properly?p=b0dffe56) I said I'd test it properly and post whatever came back. This is part one.

Full Report: https://claude.ai/artifact/VzxhC95Q6dhuBBAnsMxjSg

WHAT I DID

I built 12 renovation cost estimates, about 150 rows each, and planted 24 mistakes in each one. Claude, ChatGPT and Gemini each reviewed every sheet 5 times in the normal browser chat, the way most of us paste our work in. That's 180 reviews. Then Claude fixed the sheets using different mixes of those reviews, 84 fixes in all.

Before any data came in, I wrote down the tests and what would count as "it helps." A second company had to catch at least 5 points more.

WHAT CAME BACK

Claude + Claude caught 84% of the planted mistakes. Claude + ChatGPT caught 82.4%, and Claude + Gemini caught 82%. So the second company didn't add anything here. It came in about 2 points lower, and a 5 point gain is ruled out.

On their own, Claude caught 79%, ChatGPT 63% and Gemini 63%.

Fixing barely changed with reviews at all. The fixer got 78.5% with no review and about 79% with reviews, so I can't say which reviews help fixing.

One extra I didn't plan for. If you start with ChatGPT or Gemini, adding Claude as the second look caught 12 to 13 points more than asking the same one again. That needs its own test before I'd call it a finding.

What I take from it so far... the strength of the second reviewer seemed to matter more than which company made it.

THE LIMITS

This is one kind of spreadsheet, with planted mistakes, in browser chat apps, as of October 2026. AI did the grading. Two different systems graded and agreed strongly on what counted as a catch, but no person graded it. Claude also built the sheets, planted the mistakes and did all the fixing.

After the results, I had three independent checks run on the study. The numbers reproduced, a re-graded sample agreed with 90% of the labels, and they found one bug in my answer key (8 of 288 mistakes). Dropping those 8 didn't change anything. It's all in the report.

WHAT'S NEXT

Starting with the browser chat apps was on purpose, because that's where most of us dump our work. Next I'm testing Perplexity (my usual second look, and it picks the model for you), Copilot, Meta AI, and Codex and Claude Code, which can actually run the numbers instead of reading them. After that come documents and other kinds of work. Then the harder question: is the idea better, not just more correct.

The data and code go public after the next round. One of the next tools searches the web, so I can't put the answer key online yet.

4 comments

I expected a second AI from another company to catch more... It caught less.