Shopify A/B Testing Examples: 6 Real Tests on a High-Ticket Store

I paused this A/B test after seven days. Thirteen orders without the change, five with it: conversion rate minus 62%.

The store is a high-ticket e-commerce brand. Orders are usually over $1,000 and only about two in every thousand visitors buy, so every wrong test costs real orders. In this article I open all six tests running on it right now: why I paused one, the test every CRO checklist recommends, a +137% I don’t trust, a simple size guide at +30% (and whether I’d keep it), and the one number that tells you whether you can A/B test your store at all.

Before you read on, write down your store’s conversion rate. You’ll need it at the end.

Why high-ticket testing is different

As head of growth at Karma and Luck, a nine-figure brand, I grew the conversion rate by 50% in one year of non-stop testing. I’ve worked with brands like Danielle Gerber (from zero to a multi-million-dollar brand), Pandora, and Chatelier Cosmetics (from zero to seven figures, recently acquired in a multi-million-dollar deal). For one Shopify Plus brand we rebuilt the checkout and revenue went from $400,000 to $530,000 a month with fewer visitors.

But those stores had thousands of buyers a month. This one has two in a thousand visitors. You can’t test it the same way.

The store sells premium products, many of them one of a kind. Its conversion rate sits between 0.1% and 0.4%, while the average Shopify store is around 1.4%.

How to read every test below

For each test you’ll see four things: sessions (visitors on each side, roughly equal), conversions (orders), the conversion rate, and the confidence. Under 90% confidence, I don’t call anything. My A/B testing tool hides the result until the planned end date, with one exception: an alarm from day seven when a test is clearly losing.

The six tests at a glance

#	Test	Visitors per side	Orders (A → B)	My read
1	Review in the cart drawer	~5,800	13 → 5	−62%, paused on day 7
2	Trust box under Add to Cart	~24,800	36 → 38	+6%, barely moves
3	Homepage: four bestsellers, one button each	—	15 → 16	one order; the +137% RPV is noise
4	“One of a kind” badge	—	18 → 19	+4%; judge on clicks instead
5	Illustrated style guide	~1,900	6 → 6	running, planned for 6 weeks
6	Size guide	~5,500	9 → 12	+30%, the most promising

Test 1: a review in the cart drawer (paused)

The idea: on a $1,000 purchase, a happy customer right before checkout should calm the last doubt. It sounds obvious.

About 5,800 visitors on each side. Thirteen orders without the review, five with it. It should have helped; instead it cut orders by more than half, so I paused it on day seven, before the planned fourteen.

Eighteen orders isn’t statistical proof. But this change ran on every page of the site, and on a site-wide change I don’t wait for proof when the downside is that clear. Protecting orders comes first.

Why would a good review hurt? In the cart, people have already decided. A quote about asking the team for photos and a video can put a new doubt in their head: should I ask for a video first? And it’s one more thing to read at the exact moment you want them to click Checkout.

Social proof belongs before the decision, on the product page. That’s exactly where one review added $67,365 on another store, and it’s what the next test does.

Test 2: the trust box every checklist recommends

Trust badges under the Add to Cart button are on every CRO checklist. This store’s version: certificate of authenticity checked by experts, free insured delivery in 24 hours, 14-day returns, and secure payments with interest-free instalments.

Do you think it won? This is the biggest test on the store: about 24,800 visitors on each side. Thirty-six orders without it, thirty-eight with it. Plus 6%.

Everyone says trust badges work. After a month and almost 50,000 visitors, it’s two orders. It isn’t failing; it just isn’t doing much. And look at the plan: 84 days. That’s what a product-page test looks like on a high-ticket store.

Best practices are a starting point to test, not a result. Remember that, because the next one looks like a huge win.

Test 3: the homepage, and the +137% I don’t trust

Instead of a general banner, the homepage showed four best-selling products with one button each. Did more people buy? Fifteen orders versus sixteen. One order.

But revenue per visitor was +137%. Judged the usual way, this is a huge winner.

The orders are the same, so the extra money came from somewhere else. A product here can cost a few hundred dollars or ten thousand. The average order was about $1,000 on one side and about $2,200 on the other: one or two expensive orders happened to land on the variant. With fifteen orders, one $10,000 order brings in more than the other fourteen together. One customer decides the test.

It cuts both ways. The trust box from test 2 shows revenue per visitor at −56% with the same number of orders.

So on high-ticket stores, I judge tests on conversion rate: every order counts as one, whatever it costs. Order value I watch over months, not test by test. On conversion, this test is one order and will most likely end flat. That’s a real answer too: the hero isn’t what stops people from buying.

Test 4: the “one of a kind” badge

One small label on every product that’s the last one in stock. On a high-ticket store, “one of a kind” is the strongest form of scarcity there is.

Eighteen orders versus nineteen, plus 4%: one order. Scarcity should make people act. But on a category page, the job isn’t the order, it’s the click to the product. So on high-ticket stores I’d judge this kind of test on clicks to product pages. You get hundreds of those instead of nineteen orders. (Honest scarcity has worked well elsewhere: a real low-stock counter was worth about $528K a year.)

Test 5: an illustrated style guide, and one test per page

The idea: people love this product but don’t buy because they can’t picture how they’d use it. So, an illustrated style guide: four ways to use it, and what each one says about you.

About 1,900 visitors on each side. Six orders, and six. These pages get fewer visitors, so I planned this test for six weeks from day one and will extend it until the result is solid.

There’s something most people miss here. The trust box from test 2 runs on every product page, including these. So some visitors see both changes. You’d think more tests means faster learning, but two tests on one page mix their effects. If orders go up, was it the guide, the trust box, or both?

My rule: one test per page at a time. With thousands of orders, a good tool can separate overlapping tests. With a handful of orders per side, you keep them apart.

Test 6: the size guide (+30%)

On a $1,000 product, the question isn’t “do I like it”, it’s “will it fit”. The size guide answers that right on the page: how to measure, and a simple size table.

About 5,500 visitors on each side. Nine orders without the guide, twelve with it. Plus 30%.

This is the most promising test on the store, but three extra buyers isn’t proof yet. I’ll extend it until it reaches 90% confidence and confirm it on pages without the trust box, so I know the +30% belongs to the size guide alone.

Would I keep it? Yes. It’s low risk, it answers a real doubt, and it’s the only test here where clearly more people bought. Next I’d test it on the category where fit matters even more.

Notice the pattern: the two biggest moves on this store, −62% and +30%, both came from a doubt at the moment of buying. One added a doubt, one removed it. Not design. Doubt.

How many visitors does your A/B test need?

Now the number I promised. Take the conversion rate you wrote down.

At 0.2%, proving a 20% lift takes about 200,000 visitors on each side. The biggest test on this store had 25,000, so it can only detect big changes.

If the number you need is far above your monthly traffic:

• test big doubts, not details;
• run tests for 6 to 12 weeks;
• judge on conversion rate, not revenue per visitor.

Six lessons for high-ticket CRO

1. When only two in a thousand buy, test big changes.
2. The biggest moves come from doubt at the moment of buying.
3. Judge on conversion. One expensive order can fake a +137%.
4. Pause fast when a site-wide test looks bad.
5. Best practices aren’t wins until your data says so.
6. One test per page at a time.

What I’d test first on your store

Find the biggest doubt at the moment someone decides to buy: fit, trust, shipping, or “is it worth the price”. Test the change that removes it, on one page, and let it run long enough to mean something. If you’re unsure where that doubt sits, start with your funnel numbers: the step with the biggest drop usually hides it.

These tests are still running by design, because high-ticket tests need time. When they finish, I’ll publish the final results, including whether the size guide holds at +30%.