My take: the concern is valid, but "therefore stop testing" is the wrong conclusion. The criticism has a legitimate core. The testing industry has been slow to reckon with acquisition costs. If your test "wins" on conversion rate but the winning experience makes your CAC worse, you might have won nothing — that's a real blind spot. But every single change you introduce might affect your CACs; stopping to measure their impact is not a solution. What's actually going on in most cases: paid media and on-site testing run on completely different clocks. Meta sees a 5% CAC shift and someone acts that day. A valid on-site test usually needs weeks of traffic. They're measuring different things in different places, and when paid wobbles during a test window, it's easy to blame the test. But there's something many people miss: if your numbers hurt a little while the test runs, but you walk away knowing something you'll use for the next 6, 12, 18 months... that's not a loss. That's the cost of learning something real. A temporary CAC spike that teaches you the right price point is a bargain. What I'm curious to hear from you: - Have you seen any true correlation between running tests and CAC spikes? What did you do about it? - Has this changed how you structure your program at all? - How are you handling this conversation with stakeholders? Our gut says the CAC impact is probably overstated... but that's not always how it reads in the room. How do you navigate that?

The Growth Experiment Mindset. You're probably here because you care about testing. And you're curious about what else is possible. There's a whole world of experiments beyond headlines and buttons. Pricing. Shipping thresholds. Offer structures. Bundles. These are the decisions that can truly move your business forward. And they're all testable. This community exists to help you explore that world. Not with a formula or a checklist, but with a way of thinking that helps you figure out what to test next and why it matters. What you'll find here Short video lessons on how to explore your data, design experiments, and put what you learn into action. Templates to help you plan. Examples from real brands. And a space to ask questions, share what you're testing, and learn from others doing the same. Who this is for If you're running your first test or your hundredth, if you're a founder making pricing decisions or a specialist expanding your testing toolkit, you belong here. The only thing we ask: bring your curiosity. We're here to help you discover what works for your business. Let's get testing.

Every time I test discounts I'm reminded of how hard it actually is to make discounts make sense and overcome the loss in margin. We're testing not having an offer, vs 15% off here. It increased CVR a bunch, but barely any profit gained. We'll keep testing it for longer, but just a reminder for us to be stingy when it comes to giving away margin

@Dave Diederen came on Live with Intelligems a few weeks back. He runs Syntralabs, a CRO and experimentation agency that's run a serious volume of tests across a lot of different Shopify stores. That's an enormous amount of pattern recognition to build up, store by store, test by test. So when he decided to launch his own brand, the plan should have been simple: apply everything he already knows. Instead he froze. He described it as information overload, feeling like a newbie discovering CRO again, stuck rebuilding the same page over and over because he kept imagining what a stranger would think looking at it. What got him moving again was lowering the bar on purpose. Early on, he told himself, the ad and the creative do most of the work. The site doesn't have to be the best-converting page ever built, it has to be good enough to not get in the way while the real signal comes from the ads. So he shipped it as is and let the data, not his own judgment, tell him what to fix next. There's a nuance worth sitting with here. His edge wasn't wasted, it just showed up somewhere else. All those reps paid off in the label, the copy, knowing which small change usually wins. The site was the one place his expertise worked against him, because he could see every way it might be wrong before a single visitor ever landed on it. The full interview is attached below if you want to watch the whole conversation. Curious if this lands with anyone here. Has more experience ever made it harder for you to ship something of your own, rather than easier? What's the "done, not perfect" moment that actually got you unstuck? And where does your own expertise turn into a disadvantage instead of an edge?