Claude vs Gemini for Stripe refund emails: I tested 100 angry customer replies

I spent two weeks in November replying to angry Stripe refund requests with AI-generated emails. Half were written by Claude Sonnet 4.5, half by Gemini Flash. I run customer support for three Shopify stores—a candle brand in Portland, a fitness-apparel line in Austin, and a vintage-print shop on Etsy—and Black Friday always brings the same flood: "Where's my order?", "This doesn't fit", "I want my money back NOW."

Most store owners I know either copy-paste the same canned response or spend twenty minutes per reply trying to sound empathetic without giving away the farm. I wanted to see if Claude or Gemini could write refund emails that de-escalated fast, stayed inside Shopify's refund policy, and didn't sound like a chatbot apologising to a wall.

Here's what happened when I tested both models on a hundred real angry-customer emails, why one of them kept inventing fake tracking numbers, and when Auto mode actually saved me during Cyber Monday.

The setup: same angry emails, same refund policy, two models

I pulled a hundred customer emails from the previous year—October through December, when refund requests spike. They ranged from polite ("Hi, I'd like to return this, it's not the right size") to furious ("You charged my card twice and I STILL don't have my order. Refund me today or I'm disputing with my bank").

I fed each email to Claude Sonnet 4.5 and Gemini Flash with the same prompt:

"Write a customer-support email responding to this refund request. Our policy: full refunds within 30 days if the item is unused and in original packaging; partial refunds (minus return shipping, $8.50) after 30 days; no refunds after 60 days. Be empathetic but firm. Include next steps. Don't apologise more than once."

I kept the prompt short on purpose. I've found that over-explaining tone to an AI makes it more robotic, not less. Both models had access to the Shopify order number, the customer's original purchase date, and the product name. Neither had live Stripe data—I wasn't testing API integration, just the quality of the written reply.

Claude's replies: empathetic, occasionally too soft

Claude's emails read like a patient human wrote them. Here's a real example, responding to a customer who wanted a refund on a $42 candle she'd bought six weeks earlier and already burned halfway down:

Hi Sarah,

Thanks for reaching out. I can see you ordered the Cedarwood & Sage candle on September 14th—that's just outside our 30-day return window, and since the candle's been used, we're not able to offer a refund.

I know that's frustrating. If there was a quality issue—tunneling, weak scent throw, anything like that—let me know and I'll see what we can do.

It's warm. It acknowledges the emotion without caving. It offers a path forward. The customer replied thanking us and didn't escalate.

But Claude has a weakness: it sometimes softens the "no" so much that customers think they can negotiate. In fifteen of the fifty Claude emails, the customer wrote back asking for a partial refund or store credit even after we'd said no. The phrasing—"we're not able to offer a refund" instead of "our policy doesn't allow refunds after 30 days"—left room for hope.

Claude also never invented details. It didn't make up tracking numbers, didn't claim we'd already processed a refund, didn't hallucinate a conversation with the warehouse. That mattered, because one fake detail in a customer email can turn a refund request into a chargeback.

Gemini's replies: faster, more direct, occasionally too blunt

Gemini Flash wrote emails that got to the point faster. Same customer, same burned candle:

Hi Sarah,

Your order was placed on September 14th, which is outside our 30-day return window. our policy requires items to be unused and in original packaging. We're unable to process a refund for this order.

If you experienced a product defect, please reply with photos and we'll review your case.

It's clear. It's policy-forward. It doesn't waste words. The customer didn't reply.

Gemini's tone works well when the request is obviously outside policy—wrong item ordered, buyer's remorse after two months, a return attempt on a final-sale product. It's less good when the customer is genuinely upset and the situation is ambiguous. In one case, a customer's package was marked "delivered" by USPS but she swore it never arrived. Gemini's reply started with "According to USPS tracking, your package was delivered on October 3rd"—technically true, but it made us sound like we didn't believe her. Claude's version started with "I see the tracking shows delivered, but I know porch theft is a real problem"—same facts, less adversarial.

The bigger issue: Gemini invented a tracking number in three separate emails. Not a placeholder like "TRACK123456", but a real-looking USPS format that didn't exist in Stripe or Shopify. I caught it because I double-check every email before it goes out, but if you're batching replies during a Black Friday rush, that kind of hallucination will get you a chargeback.

The side-by-side on tone, accuracy and speed

Neither model wrote a perfect email every time. I edited every single one before sending. But Claude needed lighter edits—mostly tightening the language so the refusal was clearer. Gemini needed fact-checking and a warmth pass.

When I used Auto mode and why it worked for Cyber Monday

Kryotta's Auto mode picks the best model for each prompt. I didn't use it for the hundred-email test because I wanted to compare Claude and Gemini directly, but I did use it during Cyber Monday week when refund requests tripled.

Auto routed most of the straightforward requests—"I ordered the wrong size", "This arrived damaged"—to Gemini Flash. It sent the angry, ambiguous ones—"You ruined my daughter's birthday"—to Claude. I didn't have to choose. I pasted the customer email, asked for a reply, and Auto handled the routing.

It saved me about four hours over three days. The trade-off: I lost some control over tone consistency. A customer who got a Claude reply on Monday and a Gemini reply on Wednesday might notice the shift. For a solo operator during a volume spike, I'll take that trade. For a brand with a tight voice, you might want to pick one model and stick with it.

The checklist I use before sending any AI-written refund email

1. Check the dates. Did the model get the purchase date, return window and current date right? Gemini especially will sometimes calculate "30 days" wrong.
2. Check the tracking. If the email mentions a tracking number, open Shopify or Stripe and confirm it exists. If the model invented one, delete the sentence.
3. Check the refund amount. If the email promises a dollar figure, make sure it matches Shopify's refund calculator (original price minus return shipping if applicable, minus any partial-refund deductions).
4. Read it out loud. If it sounds like a bot, it'll read like a bot. Add a contraction, cut a "we sincerely apologise", replace "Additionally" with "Also".
5. Check for fake empathy. Phrases like "We understand your frustration" are fine once. Twice in the same email and you sound like a script.

I also keep a swipe file of the best AI-written lines—"I know porch theft is a real problem", "That's just outside our return window", "Let me see what we can do"—and I paste them into emails when the model's version falls flat.

My verdict: Claude for most refunds, Gemini for the obvious nos

If I had to pick one model for every refund email, I'd pick Claude Sonnet 4.5. It writes replies that sound human, it doesn't invent facts, and it de-escalates better when the customer is angry and the situation is grey. The trade-off is that you'll spend two minutes per email tightening the language so the "no" is clear.

Use Gemini Flash when the refund request is obviously outside policy and you need a fast, firm reply. It's better at saying no without waffling. Just double-check every factual claim before you hit send, because it will hallucinate tracking numbers if you're not careful.

Use Auto mode when you're drowning in volume and you trust yourself to edit quickly. It won't write perfect emails, but it'll get you 80% of the way there in half the time.

I'm still editing every email before it goes out. I don't think I'll ever trust AI to send a refund reply unsupervised—too much money and too much brand reputation on the line. But I've cut my average reply time from eighteen minutes to six, and I've stopped dreading my inbox every morning in November. That's worth the subscription.

Questions people ask

Can I connect Kryotta directly to Shopify or Stripe to auto-reply to refund requests?
Not yet. Kryotta has a Shopify connector for product descriptions and a Gmail connector for drafting emails, but you can't set it up to auto-reply to refund requests. You'll still copy the customer email into Kryotta, generate the reply, and paste it back into Shopify or your email client. (Honestly, I'm fine with that—I want a human in the loop for anything involving money.)

Which model is cheaper for high-volume support?
Gemini Flash is faster and uses fewer tokens, so if you're writing a hundred replies a day, it'll cost less. But if those replies create more back-and-forth because the tone is too blunt, you're not saving money—you're just deferring the work. I'd rather pay a bit more for Claude and close the conversation in one email.

What if the customer threatens a chargeback?
Neither model handles chargeback threats well. Claude will try to de-escalate with empathy, Gemini will restate the policy, but both will sound too soft or too robotic. If a customer says "I'm disputing this with my bank", write that reply yourself. Explain your policy clearly, attach proof of delivery if you have it, and keep the tone neutral. Save the email—you'll need it when Stripe asks for evidence.

Does this work for refund requests on Etsy or Instagram Shop?
Yes. The platform doesn't matter—the models are writing the email, not pulling data from Shopify's API. I tested these on Shopify orders because that's what I had access to, but the same prompts work for Etsy messages, Instagram DMs, or plain email. Just make sure you're pasting in the order details (date, product, price) so the model has the context it needs.

If you're handling customer support for a Shopify store or Etsy shop and you want to test Claude and Gemini side by side without paying for two separate subscriptions, try Kryotta—both models are in the same workspace, and you can switch between them with one click. I use it because I got tired of juggling tabs during Black Friday, and because Auto mode makes the choice for me when I'm too tired to think about tone.