I used Claude and Gemini to write 40 M-Pesa refund messages: which one customers actually replied to

I sell phone accessories on Instagram and take M-Pesa payments. Most transactions go fine, but when something breaks—customer sends the wrong amount, the payment times out, the reversal takes three days—I need to send a message that explains what happened, tells them when they'll get their money back, and doesn't start a twenty-message argument in my DMs.

Last month I tested Claude Sonnet 4.5 and Gemini Flash on forty real M-Pesa refund scenarios. I split them down the middle: twenty messages written by Claude, twenty by Gemini, all sent via WhatsApp Business. I tracked reply rates, how many customers came back confused or angry, and how long it took to close each case. The difference wasn't huge, but it was consistent enough that I now pick one model for straightforward reversals and the other when the situation's messier.

Here's what each model did well, where each one tripped up, and when I'd use which.

The forty scenarios: wrong amounts, failed transactions, and slow reversals

I pulled forty M-Pesa refund cases from the previous two months. They broke down like this:

• Overpayments: Customer sent KSh 1,500 instead of KSh 1,200 for a phone case. I need to refund KSh 300.
• Failed transactions: Payment left the customer's M-Pesa but never hit my till number. Safaricom's already reversing it, but the customer doesn't know that and thinks I have their money.
• Delayed reversals: I initiated the refund two days ago, but Safaricom says it can take up to five working days. Customer is asking where their KSh 2,400 is.
• Wrong till number: Customer sent money to an old till number I don't use anymore. I can't refund it—they need to call Safaricom—but I have to explain that without sounding like I'm dodging responsibility.

I wrote the same prompt for each case: explain what happened, when they'll get their money, and what (if anything) they need to do. I alternated models—odd-numbered cases to Claude, even-numbered to Gemini—and sent the messages as-is, no edits.

Claude: polite, clear, but sometimes too formal for a WhatsApp shop

Claude's messages were consistently polite and well-structured. It opened with an acknowledgment ("I can see the payment of KSh 1,500 came through"), explained the issue in one or two sentences, then gave a clear next step. Tone was warm but professional—closer to how a Jumia customer-service agent would write than how I usually text my customers.

Where Claude worked well: overpayments and straightforward reversals. When a customer sent KSh 2,000 instead of KSh 1,800, Claude wrote: "Hi Jane, I've received your payment of KSh 2,000 for the earphones. The correct amount is KSh 1,800, so I'll refund KSh 200 to your number ending in 4523 within the next hour. You'll get an M-Pesa confirmation as soon as it's processed."

Sixteen out of twenty customers replied with a short "okay" or "thanks" and didn't follow up. Two asked when exactly the refund would arrive—I sent a one-line answer and that was it. One didn't reply at all, which I'm counting as success because the refund went through and they didn't complain.

Where Claude stumbled: failed transactions where the customer is already stressed. One case involved a customer who'd sent KSh 3,200 for a charger and power bank, but the payment failed on Safaricom's side. The money left their account, didn't reach me, and Safaricom's reversal process was going to take three days. Claude wrote a technically accurate message explaining that the transaction had failed, the reversal was automatic, and they'd see the money back in their M-Pesa within 72 hours.

The customer replied: "So you're saying you don't have my money but I still have to wait three days? This is not okay."

I ended up sending a follow-up explaining that I genuinely didn't receive the payment—it was stuck in Safaricom's system—and offered to send the items anyway if they wanted to re-send the payment once the reversal cleared. That calmed things down, but Claude's first message didn't land the empathy.

The other issue: Claude sometimes wrote messages that were a bit long for WhatsApp. One message ran to four sentences with a subordinate clause, and while it was perfectly clear, it didn't match the quick back-and-forth tone most of my customers expect.

Gemini: faster, more conversational, but occasionally vague on timing

Gemini's messages were shorter and felt more like something I'd actually type in a WhatsApp chat. It used contractions, led with the fix rather than the explanation, and matched the informal tone most Instagram sellers use. "Hi! I can see the KSh 1,500 came through—I'll send back the extra KSh 300 in the next 30 minutes. You'll get the M-Pesa message as soon as it's done."

Where Gemini worked well: failed transactions and customer panic. In the same KSh 3,200 failed-payment case, Gemini wrote: "Hi Peter, I can see the payment didn't reach my till, which means Safaricom will reverse it automatically. You should get your KSh 3,200 back within 3 days—it's already in their system, not stuck with me. If it takes longer than that, let me know and I'll help you follow up."

Customer replied: "Okay, thanks. I'll wait." No argument, no follow-up. The tone made it clear I wasn't holding their money, and the offer to help follow up if the reversal took too long reassured them.

Gemini also handled delayed reversals better. When a customer asked where their refund was two days after I'd initiated it, Gemini wrote: "Hi! I sent the refund on Monday, but M-Pesa reversals can take up to 5 working days. It should land by Thursday at the latest. If you don't see it by then, message me and I'll send you the transaction reference so you can follow up with Safaricom."

Eighteen out of twenty customers accepted the message without pushing back. One asked for the transaction reference immediately (I sent it), and one didn't reply.

Where Gemini stumbled: overpayments where precision matters. In one case, a customer overpaid by KSh 450, and Gemini wrote: "I'll refund the extra amount shortly." No mention of the exact figure. The customer replied asking how much I was refunding, which added an extra message to the thread. It wasn't a disaster, but Claude's habit of stating the exact amount upfront avoided that confusion.

The other issue: Gemini occasionally used phrasing that was too casual for customers who expect a bit more formality. One message opened with "Hey, no worries!" when explaining a failed transaction, and the customer replied with a slightly annoyed "This is not 'no worries,' I need my money back." I had to send a follow-up that was more measured.

When each model gave a wrong answer (and how I caught it)

Claude once suggested I could "cancel the M-Pesa transaction and re-initiate it," which isn't how M-Pesa works—you can't cancel a completed transaction, only refund it. I caught it because I know the platform, but if you're newer to M-Pesa workflows, you'd want to double-check any procedural claims.

Gemini once wrote that a reversal would take "1–2 business days" when Safaricom's policy is up to five. The customer got their money on day four and messaged me annoyed that it had taken longer than I'd said. I now add a buffer when Gemini gives a timeline—if it says two days, I edit to "up to five."

The numbers: reply rates, follow-ups, and resolution speed

Here's how the forty cases broke down:

Gemini closed cases slightly faster and got fewer follow-up questions. Claude's messages were more thorough, which sometimes meant customers didn't need to ask clarifying questions, but occasionally made them feel like they were reading a policy document instead of a quick chat.

When to use Claude, when to use Gemini

I now pick the model based on the scenario:

Use Claude Sonnet 4.5 when:

• You're refunding a specific amount and want to state it upfront (overpayments, partial refunds).
• The customer expects a professional tone (repeat customers, higher-value transactions).
• You need a message that covers all the details in one go so the customer doesn't have to ask follow-ups.

Use Gemini Flash when:

• The transaction failed and the customer is already stressed or confused.
• You want a quick, conversational message that matches the informal tone of WhatsApp or Instagram DMs.
• You're dealing with a delayed reversal and need to reassure the customer without sounding like a robot.

For my phone-accessories shop, I use Gemini for about 70% of refund messages because most of my customers expect a quick, friendly reply. I switch to Claude when the transaction is above KSh 5,000 or when I'm dealing with a customer who's already sent two or three annoyed messages—they usually respond better to Claude's more measured tone.

How I set up the prompts in Kryotta

I keep two saved prompts in Kryotta: one for Claude, one for Gemini. Both include the customer's name, the transaction amount, and what went wrong. For Claude, I add "write a clear, professional message" and specify the exact refund amount and timeline. For Gemini, I write "keep it short and conversational" and let it lead with the fix.

I paste the AI's reply into WhatsApp Business, read it once to check the timeline and amount are correct, and send. Takes about thirty seconds per message. On days when I have five or six refund requests, that's the difference between spending an hour on customer service and spending ten minutes.

Questions people ask

Which model is faster to generate the message?
Both take about three seconds in Kryotta. Gemini's messages are shorter, so they're slightly faster to read and check before sending, but the difference is maybe ten seconds per message.

Can I use these for Airtel Money or bank transfer refunds?
Yes. The same logic applies—Claude for precision and detail, Gemini for speed and tone. You'll just need to adjust the timeline in your prompt (bank transfers usually take longer than M-Pesa).

Do I need to edit the AI's message every time?
I edit about one in five. Usually it's to add a sentence if the customer's particularly upset, or to adjust a timeline if the AI's estimate is too optimistic. The rest I send as-is.

What if the customer replies in Swahili?
I paste their reply back into Kryotta and ask the model to reply in Swahili. Both Claude and Gemini handle it fine, though I check with a Swahili-speaking friend if the phrasing sounds natural—I've caught a couple of awkward translations.

If you're handling M-Pesa refunds and spending too much time writing the same explanations in slightly different words, try running a few through Claude or Gemini in Kryotta. Pick the model that matches your tone, check the timeline and amount, and send. Your customers get a faster reply, and you get your evening back.