Ecommerce A/B Tests That Won (and the Best Practices That Lost) | EP 40
What should DTC brands A/B test to increase revenue per visitor?
Test the variables that change how the offer is perceived before you test reassurance. In Brad Ploch's recent split tests, cutting a $279 product's sale price from $229 to $199 lifted revenue per visitor 44%, adding six problem-focused questions to a quiz doubled it, and swapping the post-purchase offer took upsell revenue per visitor from about $2 to $10. Trust signals went the other way: a bolder warranty line cut conversion 7.5%, full reviews lost 10% of profit per session to an AI summary, and USP blocks at checkout changed nothing.
The ecommerce A/B tests that paid off this season all changed how the offer is framed: the price point, how long a quiz lets shoppers sit in their problem, how plainly a headline states the USP, and where Meta sends the traffic. The "trust signal" best practices most brands assume are safe (a louder warranty, more reviews, reassurance at checkout) lost or did nothing. Andrew Foxwell and Brad Ploch walk through more than a dozen real split tests from DTC brands, with the numbers behind the winners and the losers.
We discuss:
- Why dropping a $279 product from $229 to $199 lifted revenue per visitor 44% and more than covered the margin hit
- The six extra quiz questions that doubled revenue per visitor, and the switcher review that cost 8%
- Why these "trust" signals ultimately backfired
- How a plain USP headline beat ALL of the clever ones during a collection page headline test on $2M in revenue
- Why USP and review blocks at checkout changed nothing
- One ad, five landing pages, one CBO: how weekly spend doubled & efficiency improved
- The post-purchase offer swap that took upsell revenue per visitor from about $2 to $10
- Longer product pages for premium products, and preorder vs. sold out
Subscribe to never miss an episode.
Get the newsletter: https://www.scalabilityschool.com/scalability-newsletter
This episode of the Scalability School podcast is sponsored by NorthBeam and they just launched Northbeam Incrementality. Northbeam Incrementality gives you easy, automated, self-service incrementality tests, while protecting you from the major mistakes so many people make while running incrementality tests. Your MTA handles the daily tactics, your MMM guides the long-term planning, and Incrementality provides the causal truth. It’s a closed loop that allows you to scale what works and cut what doesn't. Right now when you head over to www.northbeam.io/incrementality, they’re offering Scalability School listeners 50% off unlimited tests for a year when you join. Just tell them we sent you!
To connect with Andrew Foxwell send an email Andrew@foxwelldigital.com
To connect with Brad Ploch send him a DM at https://x.com/brad_ploch
To connect with Zach Stuck send him a DM at https://x.com/zachmstuck
Scalability School is a weekly show from Andrew Foxwell, Brad Ploch, and Zach
Stuck about what is actually working right now in DTC growth. No theory, no
recycled LinkedIn takes, just the numbers and decisions behind real brands and
real ad accounts.
Chapters
00:00 Ecommerce A/B tests: what won, what lost, and why
02:06 Price testing: why $199 beat $229 by 44% revenue per visitor
07:08 How to build a quiz funnel in Claude, and the button that added 20%
12:04 "Sit in the problem longer": the quiz change that doubled revenue
22:30 Why more trust signals lowered conversion: warranty, reviews, AI summaries
33:53 Clear beats clever: a collection page headline test on $2M in revenue
38:18 Do USP and review blocks at checkout lift conversion?
41:55 One ad, five landing pages: how weekly spend doubled
47:41 How to audit your post-purchase upsell
50:54 Longer product pages for premium products, plus preorder vs. sold out
Full Transcript
(00:00) And so I'm anonymizing these numbers intentionally, but think of like high highish AOV appliance that goes in the kitchen and somebody that you interact with on a daily basis. And so we did a price test where we the price was originally the MSRP was $279 and the kind of like not business as usual, but when there was discounting running discounted down to $229. So $50 off.
(00:22) So there's there's two things going on there. It's discounted by $50 and also the 229 number. We test that against running it at $1.99, which is $80 off, so $279 down to $1.99. And the lower list price won like an insane amount. Revenue per visitor was up like 44%. That more than covered the gross margin hit that we took by doing the discount.
(00:42) So after we ran that test, that kind of became the business as usual and spend has continued to grow behind that. And now, let's take a listen to the Scalability School podcast. Welcome to another episode of Scalability School. Just Brad and I today talking about the season we're in, split test season, which is really going to be a great episode.
(01:10) Super stoked to be with you, Brad, as always. How you doing? >> I'm great. Um, missing missing Zach dearly, but he's traveling the world, I think, or something. You know, he's he's he's busy. Yeah. I don't not sure what he could possibly be busy with, you know, that small little company he's running. Um but uh >> it's not big. Yeah, exactly.
(01:28) Who knows? >> Yeah. Who knows? We don't we don't even we don't even know. We only talked to him on the podcast, so we're not, you know, we don't hear from him otherwise. Um look, uh we're talking about a bunch of tests that you have been running today. Um, tell me about let's talk through some of these tests because you did like a price test, you did a quiz test. I mean, there's a ton of stuff.
(01:51) This outline you sent me is absurd. Um, so let's go through wherever you want to start. >> Yeah. So, I tried to I tried to order them somewhat by impact, but they all there's there's some winners and there's some losers. And I think there's as much to learn from the the losers as the as the winners.
(02:08) But, um, we can kick it off with the price the price test. So, I will uh for um probably obvious reasons going to keep these anonymous, but um feel free to ask any follow questions and I'll do do what I can to to answer that. So, um price test, which we actually I talked about this when uh Brian was on the podcast um probably a couple weeks ago now when this one is coming out.
(02:26) You should go listen to that if you haven't. It's fantastic. He's a he's a beast. Um but this we we ran a price test and so I'm anonymizing these numbers intentionally, but um think of like high highish AOV uh appliance that goes in the kitchen. um and yeah, somebody that you interact with on a daily basis. Uh and so um we did a price test where we the price was originally the the MSRP was $279 and the kind of like not business as usual, but when there was discounting running uh discounted down to $229, so $50 off. So there's there's two things
(03:00) going on there. It's discounted by $50 and also the $229 number. Uh we test that against running it at $199, which is $80 off. So 279 down to$1.99 and the lower list price won like an insane amount. Um revenue per visitor was up like 44%. And that more than covers that more than covered the gross margin hit that we took by doing the discount.
(03:24) So after we ran that test that kind of became the the the business as usual u and spend has continued to grow behind that. So that like during that during that period revenue per visitor was up 44% on Intelligj split test and we've been able to scale into that uh consistently. Uh what what could cause that? I mean I think just like getting the the 279 the 229 279 less so but 229 just feels like man you're almost at under 200 bucks.
(03:50) Like can you it just feels so random. Like can you just get under the 200 bucks like would 259 have or 249 have done better cuz it's like close to 250. I I don't I'm not a pricing psychology guy. Maybe I should be, but um something about just like getting it down under the next dollar amount I think was pretty impactful. And obviously the discount went up 30 bucks.
(04:07) So like that's meaningful, too. But um I think a lot of the lift actually came from just like getting it down to below that that next number. So yeah, 40% increase. So what does this tell you about the way people price stuff generally? Do you think that there's just not enough price testing generally? And also testing a lower price plus percentage off is is like a huge part of it.
(04:30) um or what does this say to you? >> Yeah, and and call back to the episode with Brian because I think we spend a lot of time talking about offers, but I think people just don't spend enough time doing price tests. Uh and then also on top of that, I think people are scared to play with the MSRP number maybe more than they should be. Um I think what most people do for pricing is they go look at what other people are charging in the market.
(04:50) And you can't be tonedeaf to that. But um if you've built a product that has like some unique attributes about it or your marketing is is unique enough that you can position it to have um you know per perceived unique unique attributes then I think you can you can play with pricing quite a bit particularly that MSRP number.
(05:07) Um there are definitely like legal considerations for what you should be doing with MSRP and dollars off and how long you can be running discounts but um I think more people can get away with like playing with the MSRP number to make the offer more compelling as a result. And I think that that is that's not actually what we did in this case.
(05:22) But um what we could do is you know go manipulate that MSRP number even more uh and our um you know gross margin ends up being the same. One of the biggest questions that comes up inside the Foxville Founders community is around hiring. How do you know and how do you find great people that you can actually trust to do the job? That's why I want to talk to you about Propel.
(05:42) Propel has placed more than 75 employees for members of Fox Founders. Some of the hardest roles our members have been trying to fill for months, they filled quickly and efficiently. We're talking from video editors to graphic designers, media buyers, and more in Latin America to managers, directors, and heads of department and VPs in the US.
(06:01) They have done it all. They can help you build that team that scales faster. I know that this is an ad read, and I know that it's totally one of those things you're like, "Well, it's whatever. It's an ad." Honestly though, I literally use this and have used Propel and the team over at Propel for all of these things.
(06:15) They filled offshore, they filled domestic. Like I just said, it's absolutely incredible. And honestly, the founder, Leandro, used to run an ecom brand and performance marketing agency. So, he gets it. That's one of the things I've always loved about Leandro and what I love about dealing with him. I have never even heard a negative review about working with them.
(06:31) One time, somebody said it wasn't working out and Leandro immediately replaced the candidate that he had put in the position for a person. That's like the only feedback I've ever heard. The way that they work is simple. It's performance-based. You only pay a one-off fee once you actually hire someone.
(06:44) There's no retainers, no ongoing costs, and every hire is backed by a three-month guarantee. So, go to the show notes, check it out, let me know what you think. And if you have questions about this, feel free to hit me up, andrew foxville digital.com. I'm happy to talk about it. If you go to the link in the show notes, you get 10% off your first hire with them.
(07:00) It's a great deal, really special for podcast listeners. So, I look forward to hopefully helping your brand grow and scale. Yeah, I think I think that's very true and a cool call out. So So let's talk about the quiz idea. So the quiz uh piece uh go through the arc of all that. >> Yeah.
(07:17) So we ran um this one this is like the most like developed test to go through because it had a bunch of different steps throughout it. Uh and so um we basically set up a product quiz where um we only have one product on the website, right? So like it wasn't like we were actually coaching people down to like we weren't finding your product. It's not a makeup finder.
(07:36) Uh it's like we're just just it's like a psychological play of getting people to say yes or um like self-identify as what's important to them. Um and so they are feeling there there's a psychology play uh happening. So uh the way that we tested this at first and I've talked about this on the previous landing page test is we built the quiz out and then we just launched it into meta and let meta dictate spend to that thing.
(07:58) Uh and it quickly picked up spend. >> How did you build the quiz? Uh, I mean there's a bunch of different options. Honestly, I hate every single quiz builder I've ever used for the most part. Uh, so Claude for this one. >> Okay. Love it. So, you built this quiz in Claude? >> Yes. Which is a great great takeaway. Um, is like we we knew the questions that we wanted to ask in kind of the general psychology, but actually building it out, we just let Claude build it and then just put it straight into uh straight into Shopify. Uh,
(08:22) caveat there is like if you want to store the answers to the quiz, that makes it a little bit trickier and you got to do some extra work. Um, but that can be for another another episode. Um, so launch it quickly picked up a bunch of spend was comparable to the other landing page tests that we had, but we wanted to continue to um get this sucker to perform better.
(08:41) So, round one test, uh, I've talked about this a couple times, but uh, frustrating test, but it's like I pulled up the screen recordings and I noticed something about the screen recordings. People would land on the quiz page, uh, and I watched 10, 15 in a row. Nobody took the quiz. Not a single person to click the quiz. They clicked the menu to go shop.
(08:57) They clicked the footer to try and go shop. They were clicking everywhere but start quiz. And I was like, "Okay, what's going on here?" So, we added literally a button below that that said, "Shop the product name." Um, and that increased uh revenue per visitor by 20%. Cuz it just gave the people who didn't want to shop a path to go and do whatever they wanted.
(09:17) Now, what's interesting about that and I thought I you know I've thought a lot about this and I'm not sure what the takeaway is uh entirely yet but like that that business that I'm talking about they run almost no traffic through their product page because it never ends up performing better. So even though people are clicking shop now um listicles perform better advertorials perform better quizzes perform better than the PDP even though people are landing on this quiz page and they're they're clicking off to go to go shop immediately. Um, so that was that was
(09:43) round one is just literally adding a button that made allow allowed the people who wanted to go shop to go shop and then allowed the people who want to take the quiz to to take the quiz. Round two was um what I something I I'm trying to do more of is when we think about tests is like what's going to be high impact? Is testing 10 questions into the quiz going to be more impactful than the hook or the the image that people first see when they land on the quiz page? Well, everybody sees that page. um not many people are getting to
(10:13) the end of the quiz. That doesn't mean that's not worth optimizing and thinking about, but trying to just get more people to actually engage and take the quiz um was something that we were thinking about. So, we um the original the original um uh kind of uh image on the quiz was kind of this like 3D graphic of the product that like showed um its components is kind of how I'll phrase that.
(10:35) Uh we changed that to a lifestyle photo. So, it's like a real photo. Like I'm pretty sure I actually took the photo in my in my in my office and it was comparing an alternative product to ours and we like had key call outs for like what was uniquely different about it. Um and importantly the the product that we were comparing it against was like a popular product.
(10:53) Like a lot of people switch from this product um when they consider when they consider this brand. Um so it was like a direct direct comparison to something that people recognize. Um but it was also representative of how other people tend to build this product. Um, so like it very quickly showed what was unique about what was unique about our product.
(11:10) That helped conversion rate on the quiz and ultimately revenue per session um go up about 14%. So that was on top of the the uh original um shop button. Uh >> wow. So just just a change in the image on the top of the page. >> Yeah, literally. And I think you could >> and Yeah. >> Interestingly, the other thing that stands out is that the photo that we ended up using was one of the top statics that we had in the ad account.
(11:35) It wasn't the only thing driving to the quiz, but that was it was a top performing static. And so the idea was people already like and click on this this ad. Um maybe it could work well on the quiz page. >> Yeah. Yeah. And it's just it was you showed the competitive product people are switching from >> and I mean obviously that that helps a ton. Okay. So So we've got that.
(11:56) We've got we've got the the adding a shop now button. We've got swapping the image uh to have a comparison image at the top. What did we do for the third one here? >> So, the nice thing about the third one is uh as you as you asked uh how do we build it? It was built in Claude. Uh and so it was very easy to kind of have this hypothesis.
(12:17) Like I was I was um I know everybody kind of like is already copying Marsman's uh quizunnel and a bunch of other supplement brands. Um and I was just like spending some time clicking through a bunch of these different um bunch of these different brands. Um our first round of the quiz had like four questions. That was it. It was very short, straight to the point.
(12:32) Um, but a lot of these other ones like let you sit in the problem a little bit longer. Like it's a little bit more diagnostic um of like what the issue is or like why it's an issue. So um uh so we spent some time researching like okay well what what are what alternatives are people considering? What else are they thinking about and like what makes them feel the problem a little bit more? How what helps them understand the solution a little bit more? And then what helps them ultimately understand like why we're the best solution for this
(12:56) product. Um and like so we we put together those questions. An easy version of this and what I was kind of alluding to with building a cloud is like you could just go to cloud and say like hey I've got this quiz uh can you help me sit in the problem and uh in in the problem for a few more questions like just add a couple of questions now maybe you want to feed it like customer data maybe you have some post-purchase survey data maybe you have some reviews you have some other insights that you can you can share but just like tell
(13:19) Claude to add a few more questions to help it sit in the problem longer and I think that would probably get you pretty far uh and I don't remember oh okay so conversion and revenue per visitor for this that doubling the performance. So 100% increase in revenue by by making the quiz six extra questions. >> Okay. So it's six extra questions.
(13:38) And when you say having it sit in the problem longer, I mean I have an idea what that is, but like what are you prompting to to to do to do that? >> Yeah. I mean I think literally you could say sit in the problem longer and Claude will be able to figure it out because it'll it'll have the context of like what your quiz is already saying.
(13:54) Uh, and it'll do a fantastic job of just like, okay, your your question question two says, um, are you tired? Okay, you say yes. On a scale of 1 to 10, how tired are you? Um, it's uh it's it's 8 out of 10. Okay, and then it's going to ask two more follow-up questions to that. Uh, that's going to allow you to be like, man, I really am tired.
(14:12) Yes, that you know what, I do I do act out the next day because it makes me makes me really bad. Uh, and yeah, I would I would like to I don't want to yell at my kids because I'm not sleeping. I'm making this up, but like it just it forces the person to like again kind of that psychological play of just like making people like sit and think about it a little bit more.
(14:30) And it's like we know what they're going to answer. We're just we're forcing them to read through and reconcile with that answer. Um what it reminds me of is like you can set up automation to like send you your your business report numbers every single day. But you know what else is you know what's better than that is like going into your dashboard and looking at it and saying, "Okay, yeah, yeah, that's a problem. I got to fix that.
(14:48) " or like having to type it out to somebody and saying, "Yep, we're down 20% today." Um, so yeah, just sitting in the wrong order. >> So sitting in it. So you're giving it to So just making sure I understand. So you're giving this to Claude, the quiz. It's saying saying, "Look, like what other things do we need to add to this to get into psychology? What what's the prompt that you're using to like give you better questions that you added on?" >> Literally just in the like that that's it. Sit sit in sit in the problem longer
(15:15) and like I'll let it propose options. like I won't I won't just use um I won't just use what it has but like give me give me 20 additional questions I could add to help sit in the problem longer um and it'll come up with a handful of decent ones and then you just pick the ones you like the most >> and the ones that it came up with what were different about them they were just sort of doubling down as you said like they were just like sort of making you reflect on it further and then making you be like like lining you up and
(15:42) readying you for the fact that you probably do need a product to help because you're very in need of this cuz you have sleep issues or whatever it is. >> Yeah. So, think of like like like let's say you have like three USPS um you have three things that people buy your product for like three you know maybe maybe the first one is like the most important thing and then it kind of progressively goes down from there.
(16:02) Um >> the previous quiz was like it asked one question about each of those things which was like basically like um do you have this problem? Um, and then the the longer version went from do you have this problem to like does this problem cause you this uh and what would you like to be the outcome instead as opposed to just like do you have this problem uh and then moving on.
(16:24) So just like it just progressively pushed that person down the understanding of maybe it was like the forced reflection uh a little bit of like realizing the the problems that they experience from uh from uh from not having uh the problem or product as the solution. >> Nice. Okay. Interesting. Okay. So, that doubled. So, that helped.
(16:42) >> So, the people So, and what percentage of people are taking the quiz of of the people that land on the page at this point? >> Yeah, it's a good question. I actually have no idea. Um I don't I don't know for this one. I have to go back and look, but I'm not sure off the top of my head. >> Okay.
(16:55) So, 20% are taking it or something, but but it doesn't The conversion helps because they're looking at these questions even if they're not taking the quiz. Like, can you is it a progressive quiz or can you see the whole quiz from the page? >> Yeah, progressive. You just go one question at a time. It tells you there's a breadcrumb for like how many questions like it says what you are on out of 10 which I think is important so people don't feel like they're in a never- ending quiz but there's a progress bar and it tells you how many.
(17:16) >> And can they can they look at the quiz before they take it? Like can I look at question eight even if I haven't done question seven? >> Mm- No, you got to go one by one. >> At any point at any point it shows up in a different spot on the quiz. I'm looking at it right now. Um I would love to share it but I can't.
(17:33) Uh at any point you can still click off to go shop the product though. Like there's a little bit of a menu at the top that says like go shop this instead instead of like continuing to take this. >> Interesting. Okay, so conversion and revenue per visitor doubled. Love it. So round four, you did what to make this even better.
(17:47) >> So um we have a question in the quiz which is like what are you using now? Um and so after somebody said I'm using X, we presented a review from our actual customer database where somebody switched from that product. Um, now there's a couple things that I'm thinking about in real time as we're as we're saying this.
(18:09) Um, but uh that didn't work like to get to get to the end of it. It's like you said, "Hey, you're using you're using this product." Um, we said, "Okay, here's somebody who who who loved uh who switched and used this uh instead." Um, and so yeah, it uh it actually made it worse by about 8%. So, we turned that off.
(18:25) So, I don't know if that >> added too many steps to the quiz, like did it add friction? Um, the other thing that I now now that I'm thinking about is was it too early? Like we asked that question very early on. It's like what are you using now? Um and so is it too early to say, oh yeah, like you should totally switch.
(18:40) Like this person loved it. You should switch. It's like well they don't even understand what their problem is at this point. So would it be better to maybe move the brand towards the back end? Um and then uh present that review once they're a little bit more convinced that they need a solution. Maybe. Um maybe that's a maybe that's a V6 of this test I need to go run.
(18:57) But um yeah, TLDDR it uh went backwards. >> Okay. Well, hey, so these are some good ideas. So this worked. So what do you think are other things that you'll you'll test next as it relates to this? Like as I mean what other things that you know I mean switching the image changed it significantly like >> Yeah. >> I mean are you building this thing for mobile? Like is it does it look great? It looks great on mobile I assume.
(19:18) Like >> it looks terrible on desktop. It looks great on mobile though. >> Yeah. Well that's that makes sense. Yeah. So you're building it for mobile specifically. Um load time. Let's see. What other things are you going to test, do you think? >> Yeah, I think we could make it a little bit more visual throughout.
(19:35) Like there's no um there there are steps along the way that aren't questions that like pause and like tell you like here's here's it gives you a response based on what you've said so far that helps you like we understand you. Uh we know what you're going through. Um so like it's not asking forcing you to ask a question.
(19:52) But I think on those pages we could probably do a little bit more of making it visual to help um maybe on the question page but also on those like in between pages just like add a little bit more of uh uh under quick understanding like there's a lot of I it's not a lot of text it's a quiz with you know very short answers but um you know people get distracted very easily so like can we add some more visuals throughout this that make it um uh make it work better.
(20:13) The other thing that I that I like is just like making the quiz maybe a little bit more uh persona specific. Like now that we know what people care about, can we make the quiz entirely about one USP or do we need to coach people through like here are all the things that you care about? So I said three earlier like should the quiz just be about one? Should it be about all three? Um that's the nice thing about quizzes is it's like it's progressive and personalized to the person that's taking it.
(20:40) So, you kind of have the flexibility to to just like jam all your USPs into it and it's still a build your own uh build your own adventure. Um like I had Goosebumps back in the day. Um but um yeah, I I'd be curious like what other people are doing. So, if you're if you're if you're a if you're a quiz guru, please uh tell me what you've done that uh changed your quiz because um quiz funnels have been uh super fun to dig into recently.
(21:01) >> Yeah. Cool. Okay, that's great. Okay. So now you have another testing stuff that you ran through which is the trust bucket uh is what you're calling this uh four tests one lesson more trust signals did not mean more conversion. Tell us about that friends. Quick break. This episode is brought to you by Northbeam the marketing attribution platform that we love over here at scalability school.
(21:28) If you're a marketer I mean you probably agree right? Incrementality testing is broken. They these tests require tons of labor, spreadsheets, statistical reference and the designing of a test is slow and errorprone. How are you supposed to know what to test in the first place? Right? There's a million factors that can make incrementality testing difficult.
(21:47) But the good news is North Beam has completely change the game and they fixed this. Only North Beam incrementality automates your incrementality test design and monitoring while protecting you from the major mistakes that plague incrementality testing. Northbeam builds tests using your own MTA data, launches them directly in ad platforms, and automatically monitors them throughout.
(22:07) If something threatens validity, you're alerted before it ruins the result. With Northbeam, you can launch tests in platform with a single click, saving you tons of time and reducing any chance for errors. And when the test concludes, your results flow straight into your MTA, meaning you can easily see results alongside the rest of your data.
(22:28) Incrementality was broken. Northbeam fixed it. >> So, um, trust is maybe not like the best way to phrase this, but that's like kind of the summation that I can think of with this, but um, uh, trust, guarantees, like those kinds of things. So, they all kind of fall found into this fell into this same bucket, which was like a product page test.
(22:45) So, this is all on a specific PDP. So, test number one, um, we the the product that I'm thinking of has a warranty that is like industryleading. like it is substantially better than anything else that you can get. It is like it's an expensive product. It means it means a lot to the customer or our perception was that it means a lot to the customer.
(23:06) So it shows up on the product page but um it didn't show up on mobile until after the price after the add to cart and like somebody may have already made a decision that like ah I don't know how I'm feeling about this. Uh so it was a it was a bit of a progressive test where we had uh three cells.
(23:23) So um first was no changes. Um uh second well actually may have been four cells uh but I think only three were interesting. Uh so normal product page. The second one was uh under the price it said free shipping and we added plus xyear warranty. We added the warranty information right there to make it super visible.
(23:40) That was step one. >> And step two um uh step two we just like bolded it uh like to make it way more obvious cuz it was kind of like faint like you couldn't actually see it very well. Um and so it just like wasn't super obvious that it existed. We just like tried to like put it in people's face like, "Hey, uh, free shipping plus we have an industryleading warranty that you should be really pumped about.
(23:58) Um, and it's on a product page." Um, and uh, adding that warranty information up actually dropped um, conversion rate by uh, 7 and a half% uh, compared to the the other variant. Um, I honestly I have zero hypothesis for why. Like I don't get it. Um, it doesn't feel like it was too early to say it.
(24:18) Um, so, uh, you know, people are on a product page, they're trying to make a purchase decision. You would think they'd want more affirmation of the fact. Um, but, you know, maybe at the same time they're they're they're saying like, "Hey, this is an expensive product." Like, I just expect that you're going to deal with my problems if I have them, no matter what the warranty is or isn't.
(24:35) Um, or maybe the warranty wasn't explained very clearly there. Uh, maybe we could add like an info icon that kind of explained it all. Um, I don't know. But, um, yeah, I dropped it dropped it by 7 and a half%. Uh >> yeah, I mean it's interesting like uh like you know you so I think warranty is and I was talking to another member about this recently because they had been experimenting with a warranty with their product and the messaging around that and basically my thought was you know warranty sometimes has been diluted by a lot of brands in terms of like what
(25:03) a warranty actually means and it means that you have to follow up it means that you have to you know do this you know you have to send in the issue to an address and you know sometimes it means more so it als also can signal maybe it's like has a little bit of a ne negative signal. >> So it might might be kind of interesting like um there this particular brand had had done a lot with the word protect.
(25:25) So basically um protect my purchase was what they they did and it was like um it was it was essentially a warranty but it was a protection plan. Um, and that's language that they added because they saw that, uh, Delta Airlines was using that in trying to sell insurance when they purchased a ticket.
(25:46) And so, uh, protect kind of an interesting, uh, word. And so, I think it's maybe not the warranty. The warranty makes sense, but, uh, it's also just renaming it and like, yeah, calling it the protect purchase might be something to give a shot to. Um, >> did they did they charge for that? >> They did not charge.
(26:02) It's basically it says included so it makes it sound so it's like yeah like Delta of course charges like Alons charges for this but basically they no theirs is like oh you get free product protection basically um that's >> and you know uh so you could I mean there's a lot of things you could play with in that was my point like protection is good um that was kind of a suggestion that I had made and then we talked about other phrasing um of uh functionality functionality guarantee.
(26:35) Um, uh, never stops working promise like you know. So, we were like playing with this kind of stuff of like like uh a no no worry guarantee um uh you know zero zero issue or like uh what was the thing that we >> No questions asked or something like that. >> Yeah, no questions asked. Like it was it we basically were playing around with these types of language and protect was the one I want.
(26:58) So >> that's crazy. It's like it's insane how much like small wording changes can make such a meaningful difference like protection plan. I'm going to go test that right now cuz it's like yeah it's like I wonder how many of those things throughout like different pages or ads you just like don't think about but like inverting it slightly um makes a huge difference. That's cool. I like that.
(27:19) >> Cool. Yeah. I'm glad. Yeah. I mean it's interesting little changes. I mean everybody's got looking at these landing pages that we're building so quickly, right? Like anyway, so it's like the little words like actually make quite a bit of difference. Um, so review counts, let's talk about that was another test that you ran.
(27:34) >> Yeah, this was this this also the these next two kind of piss me off. Um, cuz it just like it's things that you feel like ah this is a no-brainer like it just makes sense like this should just win. Um, and so both uh both losers in this case. Uh, and so, um, we noticed that the review count on the homepage, um, for a particular brand was like low, like the review count was in the thousands, but the the review rating was like pretty low.
(28:02) Like it was it was close to four. Like it wasn't close to five. It was closer to four than it was to five, but they had a ton they had, you know, tons of reviews. Um, and that was prior to like some product and branding changes where they were pulling at everything. like it was it was all of the products all time.
(28:18) Um including products they didn't even sell anymore. But if you remove the products that had issues and they didn't sell anymore. Um you go to the actual reviews of the product, it was like way closer to five uh like a like a point4 jump in the overall rating. Um but they had less reviews. So it went from uh mid mid thousands to a couple uh a couple thousand um like a meaningful meaningful drop like a probably like a more than a 50% drop in the quantity of reviews.
(28:42) Um but there's like a substantially higher rating. uh and the the quantity of reviews at this lower star rating um uh one uh well I guess I let me invert that the um the higher rating less reviews had 16% less revenue per session um by doing that. Um so yeah, I'm sure there's been a lot of like testing to be done on this.
(29:06) Like if you go to if you go to Google My Business, you're not going to you're not going to you're not going to call the guy who has you need your toilet fixed immediately. You're not calling the guy who has one fivestar review. You're calling the person that has 4.4, but they've got 7,000 reviews. Um, yes. But it just felt like a big it felt like a big difference to me.
(29:22) Like it was it was meaningful. Like it was uh if I remember correctly, it was like a 4.1 versus a 4.6. Like it's a that's pretty big, right? It's a pretty big jump. Um, so totally it's a huge jump, but it's validated that it's, you know, I think people are skeptical when it's perfect ratings, right? Like it's just doesn't not real.
(29:41) And so people are like it's going to be better if it's like a real number of if it's like over a four. It's like yeah, you know, like if you're we're all so used to looking at fivestar ratings now um across so much of what we do, right? Like if you look at a restaurant um and it has a 401 rating, you're like, I don't know, you know, but like I'll give it a shot.
(29:58) I mean, if it's kind of your choice, but if you do enough in the landing page to convince, like that, you know, that's pretty powerful. So, uh you know, I yeah, there's there's a lot of nuance there, of course. But yeah, it's uh I think I think it it has the the number has a lot of the and making it as real as you can is actually quite powerful.
(30:18) >> Yeah. >> So the review widget is an excellent. >> So it's very interesting you say that because um the realness is like okay so I was looking at this next one and on the product page there is um there's a section that's uh the review widget um and we had an AI summary version. There was no, you couldn't see a single actual review. It said here's the reviews.
(30:40) Uh you could see you could see the total review rating overall, the quantity of the reviews, and you could see the individual star rating. So it's like how many five stars, how many four stars, whatever. And then like a two to three sentence AI summary, like very very short with like here's what gen here's what people generally say.
(30:56) Um and I I looked at this and I'm like I just want to see like I want to see pictures. I want to see what people are complaining about. I want to see what people like. Like I want to see the reviews and read them for myself. um because I want to make the decision based on that information. Uh and so um we swapped in a full review widget where you know like it was it showed a summary of it was it was YAPO reviews.
(31:17) Um so it's like a it's a summary of it and you can expand it and see more. Um and that was horrible. Um, it was a it was a 10% drop in profit per session from going from AI summary only where you couldn't see any actual reviews to um uh full review widget with no AI summary at all. Um, and so the question I started to ask myself from here is like well why is that? Um, and maybe the AI summary is you know people are comfortable.
(31:46) I thought that there would be an aversion to AI like I you know I think there is a general aversion but I think it's context dependent which is um when you go to Amazon they have an AI summary of reviews and Amazon normalized this this like AI review thing um and so we have a we have a we have a a step two of this going right now where um all of the variants do actually include the AI summary because it didn't it didn't win this this this test at lost um we have the AI review summary uh just the AI review summary the AI review summary But the ability to
(32:16) expand, but it shows no reviews by default, but you can expand. So it's default collapsed or default expanded with the ability to see like a handful like six reviews and then you can expand it to read more. Um and and the expanded version is losing. So it's like the other two the class one is has no difference like um like those are the same.
(32:36) Um but yeah, the expanded version uh hurts performance. So >> it's like too much information maybe or something, right? It's like people just wanting load speed. I don't know. Um it's crazy. Hm. And what's the AOV of Is it the same product we're talking about? 279 or whatever. >> Yes, this this is Yes, this is also an expensive product. Yep. >> Yeah.
(32:53) >> Yeah. >> Bouncing around a little bit on the brands, but that one is expensive one. >> Yeah, it's interesting. Um, >> so don't you think like you want to see what it looks like uh in somebody else's house? Like you want to see, you know, like you want to see what it looks like, you want to see how people are using it.
(33:06) You want to you want to read real people's experiences. Apparently not. Apparently not. >> Yeah. Apparently not. Yeah. Yeah. Interesting. I don't know. I mean, it might be maybe maybe it like once people get into the reviews, it causes more questions for them and so they're like hold held back and they're like, I don't know, you know, like I I'm going to think about it a little bit more because this person said this or this said that, you know, so it's like instead of sort of optimizing it directly for a purchase
(33:32) more more quickly. >> Yeah, that's a good point. That's a good point. Yeah, I think causing more questions is definitely could be a piece like if you you have the ability to see maybe some of the lower ratings and that's causes enough hesitation um to to wonder. Yeah, that's a good point. >> Okay, interesting.
(33:50) Okay, so we've got so we've got that uh and now now we're on to the headline test. >> There's so many I've got we've got more. We can keep going. We could we could always turn it into a part two if we need to. Um but yes, so the the headline test. So, um shout out to uh Nate Legos on Twitter. Yeah, he's been talking a ton about like copyrightiting and headline tests.
(34:14) Um and so, um this is like uh somewhat inspired by uh some of the things that he's been he's been putting out cuz I think like um go to a highra page on your website. I might be um not explaining what he does really well, but like go to a high traffic site on uh or page on your site and look at the headline um and say like is this actually descriptive enough? Is it describing my my most common customer? Like is it speaking to that person or is it just a label for what this is? So in this case um we we tested on a collection page. The collection page is
(34:45) like a pretty popular destination for this brand. A lot of traffic ends up going through the homepage and then when you click the shop, it sends you to a collection page. not a ton of SKs. Um, but like less than less than less than 10 SKs, so it's like pretty limited, uh, and not a ton to to shop from.
(35:00) Uh, and so we ran four headlines for about 3 weeks and there was about $2 million in revenue collected through this test. So it's like not not a small test. It was pretty meaningful uh, in terms of overall the control described the product like uh, very standard collection page headline, t-shirts, shorts, like that.
(35:20) That's that's you can think of it like that. It was just explained what the product was, what the category of product was. That's all all it said. Uh, two of the variants that we tested, so there was there was four total headlines. Uh, two of the variants use persona adjectives. Um, so basically what we did is we went to the customer reviews. Uh, we ran it through cloud.
(35:37) Uh, Nate actually kind of gave us tip. You run it through cloud and you say like how do people describe themselves using their actual words? Like what do they say and how do people identify? The other thing that we did is um uh there's outer signal I think has been pretty popular recently is we looked at like who are the customers? Are they um executives? Are they you know are they you know stay at home parents? Like what what are they? Um and so we used we used insights from that to write a more persona descriptive headline that spoke
(36:07) to that person. Uh and then the final variant was um >> we took the category of the product but we added a little bit of like the USPS. So like we made it a little bit more uh emotionally driven uh and and added like here's why people buy this product to it. So it went from two words which was like white shirts to uh something a little bit more descriptive about what made the product actually unique and like why people were interested in the category.
(36:33) Um which was like the persona adjectives were uh kind of versions were a little bit more copyrightiting saucy. Like they were a little bit more exciting, had a little more flare to them. Like they weren't as clear but they had a little flare to them. Uh, and this last one was just like here's what it is and like why you're going to love it. Um, very very straightforward.
(36:48) And that plain USP headline um crushed everything. Um, that last one variant 4 uh by like 3 to 5% more revenue per visitor. It fluctuated a little bit and that was on $2 million in revenue. So like 3 to 5% doesn't sound that exciting, but is over a ton of revenue like makes a huge difference. Uh, and um, yeah, that's yeah, over a three-week period.
(37:09) So um, yeah, nice little little headline test. You can just go like doom scroll uh Nate's Twitter. He puts out a ton of like examples of this. They have a great podcast uh that I'm forgetting the name on right now. Tactical and practical. Uh and Brain Driven Brands with um former uh former guest uh Sarah Leger. U great content.
(37:26) Um you can just you can just steal that right from him. Uh tell him I sent you. So yeah, banger. So what does this so I mean what does this tell you in terms of a headline test? this is this is using their words to to to sell it essentially more effectively. >> Yeah, I think you could and that's what Nate sees a lot of success with.
(37:44) Uh in this case, um if I if I look back on those two persona specific headlines, um I wish I could be more more more clear about what they what they specifically said. Um but Nate's got a bunch of great examples. Um I I think they were doing they were being cute and clever, but not clear.
(38:01) Like they weren't as clear as the one that ended up winning. um like it was just a matter of fact like this is the this is the product and the specific USP for why you're going to buy it as opposed to like here's who you are and why it matters to you, you know, like it was just it played it up a little bit too much and so I think it was um clear over clever was the the the takeaway here.
(38:18) >> Love it. So the checkout block, so I mean everybody loves a good checkout block, right? Like it's you know there is not customized enough in my opinion and this is something that Mr. Stuck has done fantastically well >> over many years and we've learned a lot from him. What did you guys do? >> I and so I went into this I was like man Shopify plus no-brainer like let's go let's go add and customize the the checkout.
(38:46) Um and we just used uh Intelligence has like a nice little checkout checkout block um test that you can run. Uh and we tried it was a three cell three cell test. Um the first was just normal checkout. The second was adding a USP block, which again just like takes the core couple reasons that people buy uh and just like puts it in the checkout.
(39:05) Uh and then the the third cell was um the USP block plus like some of the great reviews, like just reviews that were sick. Like really really good, short, to the point. People love the product, added it. Um and I was like, there's no way this doesn't win. Like this is this is free. Uh and it didn't make a difference at all. No, it literally made no difference um in any case.
(39:25) like none of them really made a huge difference uh across the board. Like um I think at at best uh it was maybe like a 2% uh swing, but it like there was no I didn't I I didn't feel like very confident in that. Um but the good news is like it didn't go backwards. Like it didn't make anything worse. So um I just liked the fact that we had USPs in the checkout.
(39:42) I just liked the fact that we had reviews in the checkout. So we ended up rolling it out anyways because there was no there's no downside to doing so. But um I just made an assumption that I was like we're going to we're going to see at least a two to three to 5% impact by doing this because people are going to get to check out.
(39:56) They're going to know like hey we've got um you know a fantastic guarantee. Um you're going to love this product. Here's a bunch of people who were obsessed with it. Um and it just yeah all those trust signals made no no difference. Um uh and I'm sure people see different things but I don't I don't know what to do with that information. >> Interesting. Yeah. It's interesting.
(40:14) It's uh so it didn't really do anything. it didn't didn't move it. I mean, I think, you know, there's there's some of these some of these where a checkout block might actually be really effective where, you know, you're pushing people through um on a quick sub uh product potentially, right? And it's and it's like if they're they're at the they're at the buy box and they're like, "Whoa.
(40:32) " And then, you know, they're like, "Wait, should I buy this?" And then it's like if there's some, you know, USP kind of pieces there, it's like, "Oh, okay. You know what?" Like, "Yeah, I'm good." And then they buy it. And like, so it might be also product dependent because it is it was this the expensive product? Yeah.
(40:46) >> Yeah. Because this it because it's like at that point they've already done the research. They're like, I don't it doesn't I don't care. You know what I mean? Like it doesn't really mean anything to me. That'd be my guess. >> That's a good point. That's a good point. >> It's like if I'm if I'm going to buy a $280 product or 275 bucks, you know, it's like I'm looking at that, I'm like, okay, like I don't it's not going to make a huge difference to me whether >> you've been through Reddit, you've been through YouTube, like you've already
(41:07) seen, you know, you it's like, all right, that's fine. you know, like I don't need to it doesn't validate actually making it faster for me to buy it is actually what I'm looking for probably, right? >> Yeah, that's interesting. Um I heard this I don't remember where I heard this but um like what it was this like story about people like having to say like oh yeah I I I promise.
(41:28) Well, it's like if you are just a person who you stick to your word, you don't have to say I promise. And so like just the just like the the emphasis of saying like oh you can trust me or I promise or whatever is just like it's the unsaid thing that doesn't need to be said. It's like, are we forcing too much into this checkout? We're like, we're like like people customers know the game.
(41:44) Like they know what you're trying to do. They're trying to butter me up at checkout, make me feel comfortable with my purchase. Um, yeah. No, that's a that I think the research thing is a really good point. I hadn't thought about that. >> Um, well, let's Okay, so now we have more destinations test. I love this. So, what do you what what does this mean? >> This one's super easy to do.
(42:06) Um, especially if you're in uh apparel. So, apparel category. Um, you tend to have like a ton of different pages that you can send people to. You can send people to a homepage if you don't have like a ton of SKs. You can send people to a collection page. You can send people to a product page. You probably have different versions of collection pages depending on how many SKs you have which like present them in different ways like fall versus summer versus whatever um or colors and different orders of them.
(42:26) Um, I think Matab um has like talked about like even the way like AB testing the way you sort collections um and the impact of that. But in general like you you kind of default just like have these pages. Uh and so for a brand we noticed that um comfort was actually running all of their traffic on meta through collections pages.
(42:48) Um and for this brand we were doing it all through product pages and like we had tested collection pages in the past and didn't really seem to make a difference. Um but Comfort recently was doing this and so I was like a okay well that's that's interesting and and worth worth um giving it a shot again. And so, um, but before we did that, um, I'm getting a I'm getting a little bit ahead of myself.
(43:07) Before we did that, we all traffic was going to the product page. We first launched, um, a listical based on the top performing ad that helped increase overall spend by 20% and efficiency held. So, like huge win there. Um, and then we added another list with some of the other top USPS that bumped up spend again a little bit as well.
(43:26) Uh, and then we launched two more versions of the collection page. Um, all the same ads. So like one ad, one top performing ad going to five different destinations within a CBO and letting like individual adsets per so we could just like control the bids if we wanted to and just letting it meta figure it out. Um >> uh there's there there was more but like the the the the most important takeaway I think is that um the top performing ad was the same across all of them.
(43:51) It still went to all those different destinations. >> And you did you copy and then put the new one in or would you just swap it? uh dup duplicate the app duplicate. Yeah. Okay. Right. Okay. So, you duplicated it, put it in the new destination >> and by by allowing and then like and it was it like any bidding nuances or anything like that they're all bit there's all the same because it's okay.
(44:14) >> So, you put it in and it and it goes through and says like we love the variety. >> Yeah. Basically, yeah. It's like you're meeting you're you're allowing meta. Okay. We talked we've talked about this a trillion times, so I'm falling on deaf ears a little bit here, but um like Andromeda is trying to do personalization at scale, right? Like they are just trying to match you to Okay, you uh you in particular, okay, here's a sick new bike.
(44:41) You're going to love this. You're going to love this bike, Andrew. You need this bike. Like, okay, you need I know you're just I you should get a new bike. >> Uh yeah, it's like they know what you're doing. They're trying to personalize that scale. Um like I I just think this extends to uh landing pages and at different points in in the shopping journey.
(44:58) So um I think this is like common knowledge, but like a lot of brands scale through quiz funnels um but then some of their lowerfunnel traffic goes through the product page. Uh and in this case we're just saying here's five options meta like you determine how to spend between them all. Uh and the net impact of this one was um like the weekly spend basically doubled uh and efficiency went up over this over this period.
(45:17) So um >> just like a huge net win. But what I would say, what I will say is like the first couple days looked really bad. Like the collection page looked horrible. Like it was not good. And I don't know if that was meta trying to figure out like who's going to prefer the collection page over the product page because it's the same ad.
(45:29) Um but yeah, once we once we gave it enough time like it it sat in and it looked really good. And it wasn't just like a week overweek thing like year-over-year. This this it was a specific product category. That specific product category is up a ton like way more than it should be for the at this point during the year.
(45:44) And like that was really the fun that was the only thing that really changed. >> I mean beautiful like great. Okay, cool. I mean, it's I mean, it's a good idea, right? It's like it's it's it's using the it's using the thinking that we know, which is to give variety and give give options. Um, and it's giving you a structure to do that.
(46:00) So, it makes sense. I somebody else uh in the membership had talked about a very similar thing where they had a bunch of different types of landers that they built. Um, and they actually had I think it was like 13 of them running or something and um and had them in a CBO, same thing. um and just let it rip on the on on a variety of top ads.
(46:19) They weren't using the same one and it it had been something that was able to continue to hold scale was their point. So very similar concepts. That's cool to hear. Um okay, love I mean we're getting whoever's listening to this is getting insane because this is good Um and also uh I I am slow on my bike right now.
(46:42) I haven't been on the bike in like I've been on the bike like four times in a month which is very unusual for me. And I went up yesterday. It was a billion degrees in Santa Barbara. It was like it was literally 100° which never is here. >> And I like went up uh the mountain and like started up with the buddy and like got to the midpoint. I'm like all right.
(47:00) I cooled off. I'm like I think I'm good. I keep going. Got up like almost got to the top and I was like I'm going to barf. Like I got to turn. I was like I I'm going I look at my GPS and I'm going three miles an hour. I'm like this is dumb. We gota like normally I'll try to aim for like 5 miles an hour which is you know because you're but you're going uphill.
(47:19) It's like a 3,800 ft climb over 7 miles but like I it's a great climb. It's very difficult but uh with the heat I was like I can't like I can't do this. And so I am slow right now. So um >> you're about to you're about to get a bunch of ads for creatine gummies and electrolytes and peptides. So >> hope you're ready. Oh yeah, I know cuz I set it into the Ether now since Meta Meta is listening, of course. Always.
(47:40) >> Absolutely. >> Um, okay. Love it. So, the post-purchase swap, what do we got for that one? >> Yeah, here's the like the big takeaway from this is like just go audit your post-purch purchase upsell. Uh, because this is like the the actual like tactical change here was not like it's not that meaningful.
(47:54) I I'll explain it anyways, but um I'm sure half the people listening to this haven't thought about the post-purchase upsell. Although Afterell, uh not a sponsor, but certainly could be, uh has just been ripping on Twitter lately given given a bunch of free game. So maybe maybe that's sparking people to go and audit their their post-purchase flow.
(48:09) But uh if you haven't looked at it, go set something up. And there's a ton of different options you could do for this. You can go upselling more of the same product. Um you can do uh trying to cross-ell products. You can try to crossell multiple products. Um you could try to upsell people into a subscription if they didn't subscribe.
(48:24) Like there's just so many options and things you could do. Um, I I really I haven't done this, but I really want to set up like a post-purchase flow where you've got like a nice little video that's like, "Hey, thanks for buying." You know, it's going to make this great is if you buy this or like it starts to like get them ready for their purchase in a way and maybe tries to upsell them on something, but actually just like it's got a sick sick video post purchase.
(48:42) Uh, maybe that helps with retention, take rate, whatever. Um, in any case, here's the test. um this brand had like a small subscription service was basically like uh hey this is like here's a complimentary subscription to to your product like you you'll just get this and it's going to help you like take care of your product but it like cost money so you have to subscribe on this but we're going to give you a great deal like you pay quarterly uh and you get a sick bundle that's going to help you take care of your your product um huge value um and
(49:07) it worked like it worked fine uh and like you know it's nice they have a little subscription revenue coming in um but they launched a complimentary product uh and I just like put an AB test in for upselling to subscription uh and upselling to this way more this like way more complimentary product um that just goes with it and the um the revenue per visitor went up like 5x.
(49:29) It went from $2 to $10. Uh and that's holding and like it, you know, realistically it doesn't make a huge difference. Like take rate and conversion take take rate actually went down a little bit but AOV just went up so much because the the product pricing was much better. Um so it didn't matter. Uh, and like the subscription revenue is not meaningful.
(49:45) Like it's not going to compound and beat that over time. Um, you know, maybe it increases from that $2 per person to $4 per person over the course of the year or even two years. The the take rate on um going to like a net new product is is a huge win. It's hard to like make that useful for people because like you have to come out with a completely new product in order to do that.
(50:04) But yeah, that's kind of why I'm saying just like go look at your post-purchase upsell and just like try something different. Even the wording. Um, we did this test one time where it was like a it was like a testing kit. Um, like you know like think of um what are the popular ones right now? Like superhuman um or whatever like a testing kit like that.
(50:20) It wasn't super human but it's like that. We added messaging in the upsell post-purchase upsell that was like hey did you know that 97% of people hit their goals when they have an accountability partner >> um like just added that messaging and had an upsell to an additional unit because like you're not going to use another testing kit like you especially not at the same time.
(50:39) Uh, and that crush like that messaging was just like helped people um, uh, think like, oh yeah, like I'll buy this and then I'll Venmo request my or like I'll get my spouse to do this or whatever. Uh, and um, that messaging made a huge difference. So, audit your post-purchase upsell. >> Okay, beautiful. Okay, so let's talk about the argument PDP as it relates to a premium watch band.
(51:00) >> Yes. So, um, pretty expensive uh, several thousand uh, watch product. Um, and we basically just took the um the the product page and like the the general premise for product page tests that I like to do. Um, and shout out to uh Kanuka who's the um CRO goat. Um, she like I I'm probably uh mixing up her words here, but um the basic premise was just like add more to the page.
(51:28) Like just add depth to it. Now be intentional about what you're adding, but like people will scroll if they want to consume more content. Like that is kind of one of the one of the bigger takeaways is like just add content to this. Uh and so um we added uh in in in the the old page it was like the the flow of the page was uh buy box.
(51:47) Hey, it includes these things for free. Um here's like your normal accordians which is like here's the warranty information blah blah blah blah blah. Um here's the the full image gallery. Um uh here's like the specs and what it's made out of. Here's how like it compares like in nighttime versus daytime.
(52:02) Here's how it looks. Um, and then all of the different um, product reviews, but the reviews were specific to only that product, not their entire collection, which matters because like they sell they sell a premium watch. They don't sell a ton of individual units, but they sell a lot across their entire collection.
(52:16) The new page um, added sitewide proof. So, that's kind of just what I just explained. Um, it added a little bit more of like it moved the the specs up a little bit more cuz these are people who are buying watches because they care about how it's built. like they want to they're like the they're like the person that overexlains their watches like dude I never ask but like they want to be able to tell you that it's Swiss made and blah blah blah blah blah cuz they're a watch collector and they care about that. Um uh it included what you get in
(52:41) the box. Um it moved moved the demo up a little bit. Uh and then it had like a comparison chart. So like well okay I got ahead of myself. It had a why no two watches are alike on it. It said why it's different. It added a value comparison like you can buy ours for x amounts or you can buy this super expensive uh Swiss alternative for tens of thousands of dollars and here's how they compare.
(53:02) Uh it showed how uh how people would wear it. Uh where it was built which it was it was made uh it was not like made overseas and that was that was valuable to this buyer. Um it made the the the risk reversal/ like warranty and guarantee more obvious. We added more depth to the FAQs. We added more reviews.
(53:16) Uh, and then one other thing that I didn't list here is like in the buy box section, you could previously like switch to a different um it was there not technically a variant because it's like a different it's a different skew uh of watch, but we just like made those much more visual like they went from um like kind of hard to see icons like bigger circles in like right at the top of the buy box.
(53:35) >> People are clicking the out of that cuz they could just like actually see like oh I think I prefer that color. Uh, and so, um, depth of page like went from like, yeah, just we just like made the page substantially longer and then changed a couple of those things throughout and just like added more information that we thought would help answer questions.
(53:51) To your point earlier about like people are going to do their research before they buy, well, great. What if it's all on the product page? >> Yeah. I mean, if you're buying like a high dollar thing, right, the more that there's there that there is there, the better, right? It's like I think you're talking about it, you're going through it, you're deeply getting into it.
(54:06) Like the more info you have and the more that you can consume and be bought into the story is watch is so much of it is the story cuz people are going to ask, "Oh, where'd you get that?" You know, like the more that you can tell that narrative is huge. And um >> yeah, I heard I heard once from another watch brand.
(54:25) Um, I worked with a watch brand for a while and they had to people that had bought their uh, watches, their watches were average about 250 to, you know, um, their they sent something like 17 emails over 45 days talking about the story and then also like constantly invited him to this Facebook group and this Instagram group.
(54:51) you could choose which one you wanted to be a part of or like a Instagram group. What was it? It was like a owners like cohort thing that they had built like anyway and it was crazy like people want to talk about it is the point. People want to be into it. People want to be a part of it and the more that you can pump them up the better off it's going to be.
(55:09) So it makes sense that it's on the PDP that way too. >> Yeah, that's cool. Yeah, I like the Facebook group thing cuz you're right. Like even if you haven't purchased yet, it's like you want to go see other people that are pumped about it. And like Facebook groups are like you can kind of say I mean I know there's moderators, but like you can kind of say whatever you want.
(55:25) So it's like you at least get the real authentic version of like what people think about it. Uh it's like uh it's like the the the thing that Zach said um probably a year ago now, which was like add Facebook comment screenshots to your landing page or product page. Like it's hard to fake that. >> Yeah, totally. Well, that's cool.
(55:42) That's interesting. So that's that makes sense. So these are I mean tons of good tests here. So now we have bonus pre-order versus sold out. Same out of stock product two framings. The hypothesis is the soldout signals demand should convert better. What happened? >> Yeah. So um you you you don't have the product.
(56:02) You can either say pre-order or sold out. You could probably do some combination of the two. But to me to me sold out says like we had this product at one point and u we had so much demand that we couldn't keep up on it. You could still buy it. like it. It said sold out, but you can buy it. Like you can be part of the next uh the next run or next batch.
(56:19) Um or you can say, "Hey, this is a pre-order." Um and I thought that the um the sold out messaging would like a little more FOMO in that, right? I don't want to miss this. Um and the pre it just made no difference. Like it just made no difference. Like pre-order worked just fine. Even if like the product had been in stock at one point, um pre-order versus sold out just made made literally no difference at all.
(56:38) Uh and so yeah, L for uh that type of scarcity uh in this case, I suppose. >> Nice. Interesting. All right. Well, look, um you know, if you listen to this, you've gotten thousands of dollars worth of advice for zero. Um thanks to Brad. Brad, anything you want to say before we close this out? >> Not really.
(56:59) Um just like there's there's so many different types of tests that you can do. So, um you know, if you don't constantly have like an intelligence test of some kind running, you should probably consider doing that. Uh, and if you need ideas, download the transcript from this and uh, rip it through CL, give you something. >> So, hopefully you have uh, enough ideas for the next year.
(57:16) >> Thanks everybody for listening. As always, we enjoy and appreciate any ratings that you give us. Have a great rest of your day. The only way that we grow this podcast is by you sharing it with your friends. Honestly, like reviews kind of don't really mean anything too much anymore. They're really meaningful, but they don't do a lot for the growth of the podcast.
(57:37) Um, and so sharing YouTube links, sharing Spotify links, sharing Apple, whatever we call it under the podcast app. Now, anything you can share, the better we're going to be. Um, guys, anything else you want to say on this? >> Yeah, please go check us out on YouTube. Rack up those views for us. We'd love to see it. And then subscribe.
(57:54) Make sure to subscribe on YouTube as well. >> And I relentlessly refresh the YouTube comments because it dictates my mental health for the day. So, please say something nice about all of us. Thank you everyone. Thank you for listening.
Made With AI
