I used 3 AI models to review 40 pull requests: which one caught the logic bugs

I had 40 pull requests from a Shopify app build and no senior developer to review them

I'm a freelance Rails developer. Last quarter I built a Shopify app for a client who runs a subscription box business—think monthly coffee deliveries with a member portal. The app handles recurring billing through Stripe, sends SMS reminders when a box ships, and lets customers swap items up to 48 hours before fulfillment. I worked with two junior developers who wrote most of the feature code while I handled architecture and client calls.

By the end of the project we had 40 pull requests waiting for review. Some were small—updating a Stripe webhook handler to log a new event type. Others were big—refactoring the subscription pause logic so customers could skip a month without canceling. I needed to review every one, but I was also building a different app for another client and had maybe 90 minutes a day to spend on code review.

I decided to test three AI models: Claude Sonnet 4.5, Gemini Pro and DeepSeek V3. I'd paste in the diff, ask each model to review it, and compare what they caught. I wanted to know which one found syntax errors, which one spotted logic bugs, and whether any of them could replace a real senior developer's eye for the stuff that breaks in production.

The test: same 40 diffs, three models, one scoring rubric

I picked 40 pull requests from the project. Ten were under 50 lines (small bug fixes, copy changes, dependency bumps). Twenty were mid-sized—100 to 300 lines, usually one feature or refactor. Ten were large—400+ lines, touching multiple files.

I gave each model the same prompt: "Review this pull request. Flag syntax errors, logic bugs, security issues, performance problems and anything that will confuse someone reading this code in six months."

I scored each review on four things:

• Syntax/style: Did it catch missing semicolons, unused variables, linting violations?
• Logic bugs: Did it spot the code that would throw an error in production or produce the wrong result?
• False positives: Did it complain about things that weren't actually problems?
• Clarity: Could I act on the feedback without re-reading the diff three times?

I didn't time the responses—all three models answered in under 30 seconds for every PR. Cost wasn't the variable here; I wanted to know which one I'd trust with production code.

Claude caught the logic bugs, Gemini caught the style issues, DeepSeek was the anxious junior

Here's what happened across all 40 reviews:

Claude found nine actual logic bugs. One was a Stripe webhook handler that would process a customer.subscription.updated event even if the subscription belonged to a different Shopify store (we run the app for multiple clients). Claude said, "This will charge the wrong customer if two subscriptions update at the same time." It was right. Another was a date comparison in the pause logic—if @subscription.next_billing_date < Date.today—that would fail if the customer's timezone was ahead of the server. Claude suggested using Time.current and comparing in the customer's zone. That would've been a silent bug for every West Coast customer.

Gemini caught 18 style issues—unused variables, inconsistent indentation, a method that could be private. It found four logic bugs, all in the smaller PRs. It missed the Stripe webhook race condition and the timezone bug entirely. It did catch a SQL N+1 query in the member portal (loading subscriptions in a loop instead of eager-loading), which was useful but not a showstopper.

DeepSeek found six logic bugs, including the webhook issue. But it also flagged 11 false positives. It said a rescue StandardError block was "too broad" when it was actually fine—we wanted to catch any error and log it without crashing the background job. It complained that a method was "doing too much" when it was 12 lines and perfectly readable. Every review felt like a junior developer trying to impress me by finding something wrong.

The pull request where Claude saved me $4,000

One PR refactored the subscription pause feature. The old code let customers pause for one month, then automatically resumed billing. The new code added a "pause indefinitely" option and a manual resume button.

The junior developer wrote this:

def pause_subscription
  @subscription.update(status: 'paused', paused_at: Time.current)
  StripeService.pause_subscription(@subscription.stripe_id)
end

def resume_subscription
  @subscription.update(status: 'active', paused_at: nil)
  StripeService.resume_subscription(@subscription.stripe_id)
end

Gemini said it looked fine. DeepSeek suggested adding error handling. Claude said, "If the Stripe API call fails, the database will show the subscription as paused but Stripe will keep charging the customer. You need to wrap the update in a transaction and roll back if Stripe returns an error."

Claude was right. If Stripe was down or rate-limited us, we'd mark the subscription as paused in our database but the customer would still get billed. They'd email us, we'd refund them, and we'd lose the Stripe fee. With 200 active subscriptions, that could've been ten angry customers and $4,000 in refunds over three months.

I rewrote it to call Stripe first, then update the database only if Stripe succeeded. Claude caught that in 20 seconds. I would've merged it and found out the hard way.

When to use which model

Use Claude Sonnet 4.5 when you're reviewing code that touches money, user data or anything that will wake you up at 3am if it breaks. It's the only one that consistently thought about edge cases and failure modes. It's also the clearest—every suggestion included a one-line explanation and a code snippet showing the fix.

Use Gemini Pro when you're reviewing small PRs and you want a second pair of eyes on style. It's great at catching unused variables, inconsistent naming and performance issues like N+1 queries. It missed the big logic bugs, but it's fast and it won't drown you in false positives.

Use DeepSeek V3 when you're reviewing code from a junior developer and you want to teach them what to look for. It's thorough, maybe too thorough. It found six real bugs, but the 11 false positives meant I had to re-read the diff to decide if DeepSeek was right. That's fine if you're mentoring someone—you can discuss why the false positive isn't actually a problem—but it's not what I want when I'm trying to ship.

What none of them caught

All three models missed a bug that a human reviewer would've seen immediately: a button label that said "Reactivate Subscription" when it should've said "Resume Subscription." The code worked fine, but the copy was confusing. AI models read diffs as logic, not as user experience.

They also didn't flag a test that was testing the wrong thing. We had a spec that checked if the pause button appeared on the page, but it didn't check if clicking the button actually paused the subscription. A human reviewer would've said, "This test is useless." The AI models said the test passed, so they moved on.

If you're using AI for code review, you still need to read the diff yourself. The AI is a first pass, not a replacement.

Questions people ask

Can I use AI to review pull requests from developers I don't trust? No. If you don't trust the developer, you need to review the code yourself or hire someone you do trust. AI will catch some bugs, but it won't catch malicious code or someone who's fundamentally not good at their job.

Which model is cheapest? DeepSeek V3 costs about $0.10 per review for a 200-line diff. Gemini Pro costs $0.18. Claude Sonnet 4.5 costs $0.35. I'd pay the extra $0.25 for Claude every time, because one missed bug costs more than a year of code reviews.

Do I need to write a custom prompt for each PR? No. I used the same prompt for all 40 reviews. The more specific you are about what you're worried about—"check if this Stripe integration handles webhooks idempotently"—the better the output, but a general "review this code" prompt works fine.

Can AI review frontend code as well as backend? Yes, but it's better at logic than design. Claude caught a React component that would re-render on every keystroke (bad for performance), but it didn't tell me the button was ugly or that the modal animation felt janky. For CSS and UI, you still need human eyes.

I'm using Claude for every PR review now. I paste the diff into Kryotta, get the review in 30 seconds, and merge with more confidence than I had when I was reviewing alone. It's not perfect, but it's better than shipping code at 11pm and hoping nothing breaks. Try it at Kryotta and see what your own pull requests look like through Claude's eyes.