Why I stopped using ChatGPT for every stage of code review
I freelance for three small Shopify agencies in the Bay Area and Austin. Most of my work is reviewing pull requests for custom storefront apps, Stripe checkout flows, or inventory sync scripts that talk to NetSuite. A typical week is eight to twelve PRs, ranging from a twenty-line bug fix to a 400-line feature branch that touches the payment logic.
For about six months, I ran everything through ChatGPT. Syntax checks, logic bugs, security audits, documentation gaps—same model, same chat window. It worked, sort of. I caught obvious errors. But I also missed a race condition in a cart-update handler that caused double charges during Black Friday testing, and I once approved a PR that logged full credit card objects to Sentry because ChatGPT told me the redaction regex looked fine.
Last month I started routing different review tasks to different models. Syntax and style to a fast model, logic and edge cases to a reasoning model, security to a model that actually explains the attack surface instead of saying "looks good." The double-charge bug would've been caught in stage two. The logging leak would've been flagged in stage three. I'm not saying I'm perfect now, but I haven't missed something that obvious since I switched.
Here's the checklist I use. It assumes you're reviewing a PR for a Shopify app, a Stripe integration, or something similar where a mistake costs your client money or trust. The same workflow works for internal tools, but the stakes are usually lower.
Stage one: syntax, style and obvious errors—use Gemini Flash
This is the fastest part of the review. You're checking for typos, missing semicolons, inconsistent formatting, unused imports, variables that shadow each other. Nothing that requires reasoning, just pattern matching against the language spec and the style guide.
I use Gemini Flash for this. It's fast enough that I can paste a 300-line diff and get an answer in under ten seconds. It catches the stuff that should've been caught by a linter but wasn't because the contributor disabled Prettier or forgot to run npm run lint before committing.
Prompt I use: "Review this JavaScript diff for syntax errors, style violations and unused variables. Flag anything that breaks Airbnb style or would throw a runtime error. Don't comment on logic yet." Then I paste the diff.
Flash is good at this because it's seen millions of lines of formatted code. It knows that const cartTotal = cartItems.reduce((acc, item) => acc + item.price) is missing an initial value, or that import { something } from 'somewhere' isn't used anywhere in the file. It's not good at understanding why the code exists or whether the logic is sound, which is fine—that's not this stage.
I tried Claude Haiku 4.5 for this stage because it's also fast, but Haiku sometimes tries to reason about the logic even when I tell it not to, which slows things down. Flash stays in its lane.
Stage two: logic, edge cases and reasoning—use DeepSeek R1
This is where I catch the bugs that matter. Does the cart-update handler lock the row before it reads the quantity? Does the refund flow check whether the charge has already been refunded? Does the discount code validator handle expired codes, codes that haven't started yet, and codes that apply to products not in the cart?
I use DeepSeek R1 for this. It's a reasoning model, which means it shows its work. When I ask it to trace the execution path through a function, I can see it consider each branch, each early return, each async call that might throw. That visibility matters because I'm not just looking for bugs—I'm checking whether I agree with the model's reasoning.
Prompt I use: "Trace the execution of this function when cartItems is empty, when one item has quantity zero, and when the Stripe API returns a 500 error. Show your reasoning for each case." Then I paste the function and any relevant context.
R1 will write out something like: "If cartItems is empty, the reduce call on line 47 will return undefined because there's no initial value, which will cause the comparison on line 52 to throw. If one item has quantity zero, the total will be calculated correctly but the shipping estimate on line 64 will divide by zero because it assumes at least one item. If Stripe returns a 500, the catch block on line 78 will log the error but won't roll back the cart lock, so the cart stays locked until the timeout."
That's the kind of breakdown I need. I can see exactly where the bug is, and I can verify the reasoning by reading the code myself. When I was using ChatGPT for this, I'd get "this function looks correct" or "consider adding error handling," which is too vague to act on.
I've also tried Claude Sonnet 4.5 for this stage. Sonnet is very good at reasoning, and it's faster than R1. But R1 shows the step-by-step trace in a way that's easier to follow, especially when the logic is nested or involves multiple async calls. If you're in a hurry, Sonnet is a fine backup.
Stage three: security and data handling—use Claude Sonnet 4.5
This is the stage where I check whether the PR introduces a SQL injection risk, logs sensitive data, exposes an API key, or fails to validate user input before passing it to Stripe. It's also where I check GDPR and PCI DSS requirements if the client sells into Europe or handles card data.
I use Claude Sonnet 4.5 for this. Sonnet is good at explaining attack surfaces and suggesting mitigations that actually work. It doesn't just say "sanitize inputs"—it tells you which inputs, why, and what happens if you don't.
Prompt I use: "Review this code for security issues. Check for SQL injection, XSS, exposed secrets, insufficient input validation, and any logging that might capture PII or payment details. Explain each issue and suggest a fix."
Sonnet will come back with something like: "Line 34 interpolates req.body.email directly into a SQL query without escaping, which allows an attacker to inject arbitrary SQL. Use a parameterized query instead: db.query('SELECT * FROM users WHERE email = ?', [req.body.email]). Line 58 logs the full stripeToken object, which contains card details. Log only the token ID: console.log({ tokenId: stripeToken.id }). Line 72 doesn't validate that quantity is a positive integer, so an attacker could pass a negative number and get a refund instead of a charge."
That level of detail is what I need. I can copy the suggested fix into a comment on the PR, and the contributor knows exactly what to change. When I used ChatGPT for this, I'd get "be careful with user input" or "consider using environment variables for secrets," which is true but not actionable.
I've tried Gemini Pro for security reviews. It's decent, but it sometimes misses subtle issues like race conditions in payment flows or fails to flag logging that's only a problem under GDPR. Sonnet is more thorough.
Stage four: documentation and readability—use Llama 3.3 70B
This is the last stage. The code works, it's secure, and it passes tests. But is it readable? Are the variable names clear? Are there comments explaining the non-obvious parts? If someone opens this file in six months, will they understand what it does?
I use Llama 3.3 70B for this. It's good at suggesting better names, identifying functions that are too long, and pointing out places where a comment would help. It's also good at writing the comment itself, which I can paste in if the suggestion makes sense.
Prompt I use: "Review this code for readability. Suggest better variable names, flag functions that are too long or do too much, and identify places where a comment would help. Write the comment if you suggest one."
Llama will say things like: "The variable x on line 23 should be renamed to discountAmount to make its purpose clear. The function processOrder on line 45 is 80 lines long and handles validation, payment, inventory update, and email notification—consider splitting it into four smaller functions. Line 67 calculates a pro-rated refund using a formula that isn't obvious—add a comment explaining that it's based on days remaining in the subscription period."
This stage doesn't catch bugs, but it makes future reviews easier. If the code is readable, the next person (or me in three months) will spot issues faster.
I tried using Claude Haiku 4.5 for this because it's fast and cheap, but Haiku's suggestions are less specific. Llama gives me the exact new name or the exact comment to add, which saves time.
Side by side: one model vs four
I ran the same PR through ChatGPT (GPT-4) and through this four-stage workflow. The PR was a Stripe checkout flow for a Shopify app that sells subscriptions. About 200 lines, touching the payment handler, the webhook listener, and the customer model.
ChatGPT found the syntax error (a missing closing brace), suggested adding error handling, and said "looks good overall." It missed the logic bug where the webhook handler didn't check whether the subscription was already active, which would've caused duplicate entries in the database. It also missed the security issue where the handler logged the full customer object, which included the email and Stripe customer ID.
The four-stage workflow caught the syntax error in stage one, the logic bug in stage two, and the logging issue in stage three. Stage four suggested renaming sub to subscription and adding a comment explaining why the webhook handler checks the event type twice.
Total time for ChatGPT: about four minutes. Total time for the four-stage workflow: about nine minutes. The extra five minutes caught two bugs that would've cost the client money or violated their privacy policy.
How I actually run this without switching tabs twelve times
I use Kryotta for this because it has all four models in one workspace. I open the PR diff in one window, open Kryotta in another, and run each stage in a separate chat. Stage one is Gemini Flash, stage two is DeepSeek R1, stage three is Claude Sonnet 4.5, stage four is Llama 3.3 70B. I copy the diff once, paste it into each chat with the appropriate prompt, and review the output in order.
If you're not using Kryotta, you can do the same thing by opening four tabs—one for Gemini, one for DeepSeek's site, one for Claude, one for Llama—but you'll be copying API keys and managing four separate accounts. I tried that for a week before I switched.
What doesn't work
Don't run all four stages with the same model. I tried using Claude Sonnet 4.5 for everything because it's good at reasoning and security. It worked, but it was slower and more expensive than using Gemini Flash for syntax checks. Fast models are fast for a reason—they're optimized for pattern matching, not deep reasoning. Use them where they're strong.
Don't skip stage two. Logic bugs are the ones that cost money. A syntax error breaks the build, so someone will catch it. A logic bug passes tests, gets deployed, and causes a double charge or a failed refund. Stage two is where you catch those.
Don't trust stage three output without reading the code yourself. AI models are good at spotting common security issues, but they're not perfect. If Sonnet says "this looks secure," I still check the input validation and the logging myself. The model is a second pair of eyes, not a replacement for thinking.
Questions people ask
Can I use GPT-4 for all four stages instead of switching models?
You can, and it'll catch most issues. But GPT-4 is slower and more expensive than Gemini Flash for syntax checks, and it doesn't show reasoning as clearly as DeepSeek R1 for logic bugs. If you're reviewing one PR a week, the difference doesn't matter. If you're reviewing two a day, the time adds up.
What if I'm reviewing Python or Go instead of JavaScript?
Same workflow, same models. Gemini Flash knows Python and Go syntax. DeepSeek R1 traces execution in any language. Claude Sonnet 4.5 catches security issues in Python (SQL injection, command injection, pickle exploits) and Go (race conditions, goroutine leaks). Llama suggests readable names in any language.
Do I need to paste the entire diff into each stage, or can I paste only the relevant part?
Paste only what's relevant. For stage one, paste the whole diff. For stage two, paste the function you're tracing plus any context it depends on. For stage three, paste the parts that handle user input, secrets, or data. For stage four, paste the functions that are hard to read.
How much does this cost per PR?
On Kryotta, about $0.15 to $0.40 per PR depending on the size of the diff and how much reasoning you need in stage two. Gemini Flash is cheap, DeepSeek R1 costs more because it's a reasoning model, Claude Sonnet 4.5 is mid-range, Llama is cheap. If you're using separate API accounts, the cost is similar but you'll pay four transaction fees.
I've been using this checklist for five weeks. I haven't missed a logic bug that made it to production, and I haven't approved a PR that logged something it shouldn't. The workflow takes longer than running everything through one model, but the bugs I catch are worth the extra nine minutes.
If you want to try this without juggling four browser tabs, Kryotta has all four models in one workspace and lets you run them side by side. I keep one chat open per stage, and I can compare the output without switching windows.
