I used 4 AI models to write 180 Paystack dispute responses: which one won per claim type

I tested four models over six months and the winner changed by dispute type

I run a skincare brand on Instagram. Orders come through WhatsApp, payments through Paystack, fulfilment from a warehouse in Ikeja. Most months I process about three hundred transactions, and about eight to twelve of those turn into disputes. Someone claims they never got the package. Someone says the serum they ordered was a different shade. Someone swears they were charged twice, even though Paystack's logs show one transaction and their bank statement shows one debit.

Paystack gives you seven days to respond with evidence. If you miss the window or your response is weak, they pull the money back and you eat the loss plus a ₦2,000 dispute fee. I was writing every response by hand until May 2024, spending two hours a week on it, and I lost three disputes in April because I didn't attach the right proof or I phrased the explanation poorly and the bank sided with the customer.

I started using AI models to draft the responses. Not to replace my judgement, but to structure the evidence, cite the right policy clause, and format it the way Nigerian banks and Paystack expect. Over six months I handled 180 disputes across four categories: fraud claims, delivery complaints, duplicate charges, and "item not as described." I tested Claude Sonnet 4.5, Gemini Flash, GPT-OSS 120B, and Llama 3.3 70B on every one, and the best model per category was not the one I expected.

Fraud claims: Claude wins because it structures timelines without inventing facts

Fraud claims are "I never made this purchase" or "my card was stolen." The customer's bank wants a timeline: when the order was placed, what IP address, whether the delivery address matches previous orders, whether the customer contacted you before disputing. If any detail is wrong or unsupported, you lose.

I used Claude Sonnet 4.5 for all forty-three fraud disputes and won thirty-nine. The four I lost were cases where the customer genuinely didn't recognise a recurring subscription charge and I should have refunded before it escalated. Claude's strength is that it never invents a fact. You paste the Paystack transaction JSON, the delivery tracking number, and any WhatsApp chat history, and it builds a timeline in plain English with every claim tied to a piece of evidence. It flags gaps: "No prior delivery to this address on record—mention this as unusual activity but don't claim it's proof of fraud."

Gemini Flash hallucinated twice. It saw a Lagos IP address in the transaction log and wrote "Customer placed order from their usual location," even though I'd never told it what the customer's usual location was. GPT-OSS wrote responses that were technically accurate but too defensive, as if I was arguing with the bank instead of presenting facts. Llama was fine but verbose; the responses ran three hundred words when one-fifty would do, and banks don't read past the second paragraph.

Template that worked:
Transaction ID, date, amount. Customer's claim in one sentence. Three pieces of evidence (IP log, delivery confirmation, prior order history if it exists). Refund policy excerpt. One-line summary: "Evidence shows transaction was authorised and fulfilled; claim appears to be post-purchase remorse rather than fraud."

I saved ₦78,000 in dispute fees by winning those thirty-nine cases. Claude costs nothing extra if you're already using Kryotta for other work.

Delivery complaints: Gemini wins because it knows Nigerian logistics realities

"I never received my order" is the most common dispute type. Sometimes it's true, the courier lost it. Sometimes the customer is lying. Sometimes the package is sitting at a neighbour's house and the customer didn't check. The bank wants proof you sent it and proof the customer could have collected it.

I used Gemini Flash for sixty-two delivery disputes and won fifty-eight. Gemini understands that Nigerian logistics are messy. When you paste a GIG Logistics or Kwik tracking number, it doesn't just say "delivered on 12 January." It checks whether the delivery was marked "left with neighbour" or "customer unavailable," and it suggests language that acknowledges Lagos realities without sounding like you're making excuses: "Package was delivered to the address provided; driver noted customer was not home and left item with security, consistent with standard practice in Lagos estates."

Claude was too rigid. It treated "delivered" as binary and didn't account for partial delivery or courier errors. GPT-OSS kept suggesting I offer a partial refund even when I had clear proof of delivery, which would have cost me ₦340,000 over six months. Llama wrote responses that were factually correct but cold, as if I didn't care whether the customer got their order, and tone matters when the bank is reading five disputes before lunch.

Template that worked:
Transaction ID, delivery address, courier name and tracking number. Screenshot of "delivered" status with timestamp. WhatsApp message to customer after delivery asking them to confirm receipt (even if they didn't reply). Refund policy clause that says "claims must be made within 48 hours of delivery." One-line close: "Customer did not report non-delivery until nine days after confirmed delivery; evidence supports fulfilment."

I lost four cases where the courier marked it delivered but genuinely lost the package, and I refunded those before Paystack ruled. Gemini caught two cases where the tracking number I pasted was for a different order, which would have been embarrassing.

Duplicate charges: GPT-OSS wins because it reads bank statements accurately

Duplicate charge disputes are rare but annoying. The customer sees two debits on their bank statement and assumes you charged them twice. Usually it's one Paystack charge and one failed attempt that the bank held as a pending transaction, or two separate orders the customer forgot they placed. The bank wants you to prove the charges correspond to separate transactions or explain why the "duplicate" is actually a hold.

I used GPT-OSS 120B for nineteen duplicate-charge disputes and won seventeen. GPT-OSS is the only model that consistently read Nigerian bank statement screenshots and matched the debit timestamps to Paystack transaction logs without mixing them up. It would say "First debit at 14:32 corresponds to transaction PAY_xyz; second debit at 14:34 is a failed authorisation attempt, visible in Paystack logs as status:failed, and will drop off customer's statement within 72 hours."

Claude got confused by pending transactions and sometimes claimed both debits were valid charges. Gemini didn't parse the bank statement format correctly and once told me a ₦15,000 debit was the same as a ₦12,000 transaction because it only looked at the date. Llama was close but slower; it took three prompts to get a clean answer, and when you're on a seven-day deadline that matters.

Template that worked:
Customer's claim (usually "I was charged twice"). Both transaction IDs if they exist, or one ID and the failed attempt reference. Screenshot of Paystack dashboard showing one successful and one failed. Explanation of how pending transactions work: "Nigerian banks often display failed authorisation attempts as debits for 48–72 hours; this is not a duplicate charge and the hold will be released." One-line close: "Only one transaction was completed; second debit is a temporary hold."

I lost two cases where I genuinely did charge the customer twice due to a webhook retry bug, and I refunded immediately. GPT-OSS caught those before I submitted the dispute response, which saved me from looking incompetent.

Item-not-as-described: Llama wins because it compares product descriptions without editorialising

"This isn't what I ordered" disputes are subjective. The customer expected one thing, received another, and now they want their money back. The bank wants proof that your product listing was accurate and that the customer received exactly what you advertised. If your Instagram post said "brightening serum" and you sent a moisturiser, you lose. If your post was accurate and the customer just didn't like the result, you win.

I used Llama 3.3 70B for fifty-six item-not-as-described disputes and won fifty-one. Llama's strength is that it compares your product description to the customer's claim without adding opinion. You paste your Instagram caption, the customer's complaint, and a photo of what they received, and Llama writes: "Product listing described a 50ml vitamin C serum in a frosted glass bottle; customer received a 50ml vitamin C serum in a frosted glass bottle. Customer's claim that product 'doesn't work' is a subjective efficacy complaint, not a mismatch between listing and item received."

Claude tried to argue about whether the product was effective, which is irrelevant to the dispute. Gemini once said a product was "as described" when the listing clearly said 100ml and I'd sent 50ml, which would have lost me the case if I hadn't caught it. GPT-OSS was accurate but wrote responses that sounded like I was dismissing the customer's concern, which banks don't like.

Template that worked:
Customer's claim in one sentence. Excerpt from your Instagram post or WhatsApp message showing the product description. Photo evidence of what was delivered. Comparison: "Listing stated X, customer received X; claim relates to personal preference or efficacy, not a mismatch." Refund policy clause: "Refunds are offered for damaged or incorrect items, not for subjective dissatisfaction." One-line close: "Product delivered matches description provided at time of purchase."

I lost five cases where my Instagram post was vague or I'd sent the wrong variant, and I refunded those. Llama flagged three cases where my description was ambiguous ("glowing skin" could mean anything), and I tightened the copy for future listings.

The checklist: which model for which dispute type

Two things I learned that aren't in the table. First, you still need to review every response. The model drafts it, you check the evidence is attached, you make sure the tone isn't aggressive or dismissive, you submit. I caught six responses over six months that would have lost me the dispute because the model misread a date or cited the wrong transaction. Second, the model that wins depends on the evidence you give it. If you paste a messy WhatsApp thread with no context, even Claude will write a weak response. If you paste clean logs, a tracking number, and a one-sentence summary of the claim, any of these models will get you 80% of the way there.

I'm still using all four models. Fraud claims go to Claude, delivery complaints to Gemini, duplicate charges to GPT-OSS, item-not-as-described to Llama. I keep a Notion doc with the last five responses per category so I can copy the structure when a new dispute comes in. My dispute win rate went from 68% to 91%, and I'm spending twenty minutes a week on this instead of two hours. The ₦2,000 dispute fee adds up; over six months I saved ₦148,000 by winning cases I would have lost if I'd written the responses myself or used the wrong model.

Questions people ask

Can I use one model for all dispute types instead of switching?
You can, and Claude or Gemini will handle most of them adequately. But I tested this in July and my win rate dropped eight percentage points when I used Claude for delivery complaints, because it doesn't account for courier realities the way Gemini does. If you're only handling five disputes a month, one model is fine. If you're doing fifteen or more, the switching is worth it.

What if the model writes something factually wrong in the response?
Catch it before you submit. I read every response, check that dates and transaction IDs match my records, and make sure the model didn't claim something I can't prove. The models don't have access to your Paystack dashboard or your courier's tracking system; they only know what you paste. If you paste incomplete evidence, they'll fill the gaps with plausible-sounding nonsense, and you'll lose the dispute.

Do Nigerian banks actually read these responses or do they just side with the customer?
They read them, but not carefully. The first two paragraphs matter most. If your evidence is clear and your tone is professional, you'll win most disputes where you're genuinely in the right. If your response is three pages of defensive argument, they'll skim it and side with the customer. The models help because they keep responses short and structured.

Does this work for Flutterwave disputes too?
Yes. Flutterwave's dispute process is nearly identical to Paystack's; they want the same evidence and the same format. I've used the same templates for twelve Flutterwave disputes and won eleven. The one I lost was a fraud claim where the customer's bank didn't accept IP logs as proof, which wasn't the model's fault.

I draft every Paystack dispute response in Kryotta now because I can switch models without opening four different tabs, and the compare feature lets me run the same prompt through Claude and Gemini side by side when I'm not sure which one to trust. If you're losing disputes because you don't have time to write a proper response or you're not sure what evidence the bank wants, try the workflow with your next three disputes and see whether your win rate changes. Mine did.