I ran the test because I was tired of guessing which model writes better subject lines
I run a home goods store on Shopify—candles, linen sprays, ceramic mugs, the kind of things people buy for themselves in October then panic-buy for gifts in November. Last year I wrote every Black Friday email by hand, tweaked subject lines until 11 p.m. the night before, and still watched my open rate sit at 19 percent while my inbox filled with newsletters hitting 34 percent. I knew the copy wasn't bad, but it wasn't working, and I didn't have time to A/B test twenty variations when I was also packing orders and answering DMs about shipping cutoffs.
This year I decided to treat it like a proper test. I picked three models—Claude Sonnet 4.5, Gemini Flash, and GPT-OSS 120B—and wrote 120 email sequences across early-bird, peak weekend, and last-chance campaigns. Forty sequences per model, each with subject line, preview text, body copy, and an SMS follow-up. I sent them to segments of my list over three weeks in October (yes, I started early; November is for executing, not drafting). I tracked open rates, click-through, and the time it took to get from blank page to "schedule send" in Klaviyo.
Here's what actually happened.
Claude wrote the most human early-bird emails and won the open rate by four points
Early-bird emails go out the week before Black Friday to people who opened your last three campaigns—your engaged list. The goal is to make them feel like insiders without sounding like you're begging. I gave each model the same prompt: "Write five subject lines and preview text for an early-bird Black Friday sale, 25% off, VIP access starts Thursday, main sale starts Friday. Brand voice: warm, conversational, not cutesy. Audience: repeat customers, mostly women 28–45, disposable income, care about quality."
Claude Sonnet 4.5 came back with subject lines like "You're in early (because you're the best kind of customer)" and preview text that named the discount in the first five words. Gemini Flash gave me "Your VIP access starts now" and "Black Friday preview: 25% off before anyone else," which were fine but felt like they came from a template. GPT-OSS 120B wrote "Early access unlocked 🔓" and "Be the first to shop our biggest sale," which I almost used until I realised every DTC brand sends that exact line.
I sent ten emails per model to three segments of about 800 people each, all pulled from the same engaged cohort so the test was fair. Claude's emails opened at 31 percent. Gemini hit 27 percent. GPT-OSS landed at 26 percent. The difference between Claude and the others was four to five points, which doesn't sound like much until you realise that's thirty more people per thousand who saw the offer.
Claude also wrote body copy that didn't oversell. It acknowledged that people get a hundred Black Friday emails and said "we're not going to pretend this is the only sale you'll see this week, but it's the only one where you get first pick of the candles that sell out by Saturday." That line came straight from the model. I changed two words.
Gemini Flash was fastest for last-chance urgency and cost $0.02 per email
Last-chance emails go out Sunday night and Monday morning of Cyber Week. You've got one job: get people who ignored the first four emails to open this one. I tested subject lines with countdown language, explicit deadlines, and a little bit of FOMO. Same prompt structure for all three models, same brand voice, same segments.
Gemini Flash wrote the best last-chance subject lines I've ever seen. "4 hours left, then it's full price" opened at 29 percent. "This sale ends at midnight—actually midnight, not marketing midnight" hit 28 percent. The preview text was sharp: "No extension. No secret Monday deal. If you want 25% off, open this now." It sounded like a person who was done being polite, which is exactly the tone you want on Cyber Monday when everyone's inbox is a warzone.
Claude's last-chance emails were good but a bit softer—24 percent open rate. GPT-OSS tried to be clever with "Your cart misses you" and "One last chance (we mean it this time)," which opened at 22 percent. The problem with GPT-OSS was that it wanted to be funny when the moment called for blunt.
Gemini Flash also generated the copy faster than the other two. I was working in Kryotta's multi-model workspace, and Flash consistently returned a full email—subject, preview, body, CTA—in under four seconds. Claude took six to eight seconds. GPT-OSS was closer to ten, which matters when you're drafting forty emails in one sitting and your toddler wakes up from a nap in twenty minutes. At Gemini Flash's pricing, each email cost about two cents in API calls. Claude was closer to eight cents. GPT-OSS was five.
I'm not saying speed is everything, but when you're a solo operator and you've got three campaigns to write, two Instagram carousels to design, and a Stripe payout that's late, those six seconds add up.
GPT-OSS wrote the best SMS follow-ups and saved me from sounding like a robot
SMS is where most AI models fall apart. The character limit forces them into generic mode, and you end up with "Hey! Our sale ends soon. Shop now: [link]" which is technically a message but does nothing. I tested SMS follow-ups for people who opened the email but didn't click. The goal was to sound like a quick text from a friend, not a Shopify notification.
GPT-OSS 120B nailed it. "Still thinking about that candle? It's still 25% off for another six hours." Or: "Fair warning: the linen spray you looked at yesterday just sold out in Lavender. We've got Eucalyptus left." It wrote SMS that referenced specific behaviour without being creepy, and it kept the tone light. Open rate on those texts (yes, SMS has open rates if you're using Postscript or Attentive) was 91 percent. Click-through was 34 percent, which is absurd for a follow-up message.
Claude's SMS felt a bit formal—"Hi [Name], just a reminder that our Black Friday sale ends tonight at midnight EST. Shop now before your favorites sell out." Gemini's were too short: "Sale ends tonight. Shop: [link]." Neither was bad, but neither made me want to click the way GPT-OSS did.
I ended up using GPT-OSS for all my SMS, Claude for early-bird emails, and Gemini for last-chance urgency. If you're writing a hundred emails and you want to mix models based on what they're actually good at, that's the split I'd recommend.
The post-BFCM thank-you emails were where Claude pulled ahead again
Thank-you emails are the ones nobody talks about but everyone should send. You write them on Tuesday after Cyber Monday, when your customers are coming down from five days of sale emails and wondering whether they overspent. The goal is to make them feel good about buying from you instead of the twenty other brands they almost chose.
Claude Sonnet 4.5 wrote thank-you emails that sounded like I'd sat down with a cup of coffee and actually thought about the person on the other end. "Thank you for shopping with us this weekend. I know your inbox was a mess and you had a lot of options. The fact that you picked us means a lot." The open rate on those emails was 41 percent, which is higher than most of my regular campaigns. People replied. One person said "this is the first Black Friday email I've read all week that didn't make me want to unsubscribe."
Gemini's thank-you emails were fine but a bit transactional: "Thanks for your order! We hope you love it. Here's your tracking link." GPT-OSS tried to upsell in the thank-you email, which I immediately deleted. Nobody wants to be thanked and then sold to in the same breath.
If you only use Claude for one thing this Black Friday, use it for the emails where you're trying to sound like a human who's grateful, not a store that's chasing another conversion.
What I'd do differently next time and what I wouldn't bother testing
I wouldn't test more than three models. I thought about adding Mistral Large and DeepSeek V3 to the mix, but by email sixty I was sick of reading my own product descriptions. Three models is enough to find a pattern. More than that and you're just procrastinating.
I also wouldn't write forty sequences per model again. Twenty would have been enough to spot the difference. The open-rate gap between Claude and Gemini held steady after the first fifteen emails; the rest just confirmed what I already knew.
What I would do again is split the work by email type instead of trying to pick one model for everything. Claude for early-bird and thank-you, Gemini for last-chance urgency, GPT-OSS for SMS. That's the workflow I'll use in 2025, and it'll probably take me eight hours instead of twelve to draft a full Black Friday sequence.
I'd also start earlier. I thought October was early. It's not. If you're writing a hundred emails, you want to be done by mid-October so you've got two weeks to schedule, test links, and fix the thing where Klaviyo randomly drops your preview text on mobile.
Cost and time breakdown for the whole project
120 email sequences took me eleven hours to draft, edit, and load into Klaviyo. That includes the time I spent writing prompts, reading the output, tweaking the ones that didn't sound right, and pasting everything into the email builder. If I'd written them by hand, I'd still be writing them now.
API costs came to about $18 across all three models. Claude was the most expensive at around $9 for forty sequences. Gemini was $4. GPT-OSS was $5. I was working in Kryotta, which let me run all three models in one workspace and compare the output side by side without switching tabs. That saved me more time than I expected—probably an hour over the course of the project.
For context, I paid a freelance copywriter $400 last year to write ten Black Friday emails. She did a great job, but I couldn't afford to hire her for forty sequences, and I didn't have time to wait for revisions. This year I spent $18 and eleven hours and ended up with better open rates than I've ever had. I'm not saying AI replaces a good copywriter—it doesn't—but if you're a solo store owner and you're choosing between "write it yourself at midnight" and "use AI to draft it in the afternoon," the choice is obvious.
Questions people ask
Which model should I use if I only have time to test one?
Claude Sonnet 4.5. It writes the most human-sounding emails across the widest range of tones, and it's especially good at early-bird and thank-you campaigns where you need to sound warm instead of urgent. If you're only sending ten emails this Black Friday, use Claude for all of them.
Can I use these models to write SMS campaigns, or is SMS better done by hand?
GPT-OSS 120B writes better SMS than I do by hand. It's good at short, conversational copy that doesn't sound like a notification. Claude and Gemini are fine for SMS, but they tend to be either too formal or too brief. If you're sending SMS follow-ups, test GPT-OSS first.
Do I need to rewrite the AI output, or can I send it as-is?
You'll rewrite some of it. I changed about 30 percent of what the models gave me—mostly small tweaks like swapping "purchase" for "order" or cutting a sentence that felt like filler. The early-bird emails needed the least editing. The last-chance emails needed the most, because urgency is hard to get right and the models sometimes oversell.
How much does it cost to run this test if I'm using Kryotta?
Around $18 in API costs for 120 sequences, plus whatever you're paying for Kryotta's workspace (pricing depends on your plan, but the free tier covers light testing). If you're only writing twenty emails instead of 120, you're looking at $3 to $5 total. That's less than one month of a ChatGPT Plus subscription, and you get to use three models instead of one.
If you're drafting Black Friday emails this week and you want to test Claude, Gemini, and GPT-OSS side by side without juggling three different platforms, Kryotta's multi-model workspace lets you run them all in one place and compare the output in real time. I used it for this whole project, and it's the reason I didn't lose my mind switching between browser tabs at 9 p.m. on a Sunday.
