The Experimentation Edge
Filter results
How Clover experiments when billions of dollars flow through daily
Why JobLeads says one test won't move you, but 100 will
Inside Aspen Dental's 100-test-a-year experimentation program
Synthetic audiences meet real A/B tests at Principal Financial Group
Why US Bank considers missing even 1% of customers unacceptable
Why Farfetch manages by learning rate, not win rate
How Cogniteer Built an Experimentation Engine From Scratch
How Fin does 1,000,000 A/B Tests in 24 Hours
How Kargo turns losing experiments into competitive edges
The 'wine effect' and other surprises that reshaped how Box runs e-commerce experiments
Diligent explains why moving on from an experiment might cost you
The metric Stitch Fix says every experimenter should chase
What the Expedia Group cannot measure, it cannot ship
Top takeaways from our favorite conversations
DoorDash's price experiment proved price by itself doesn't predict orders. Different customers want different things at different times, which pushed the team toward personalization.
Build the triad: pair an easy-to-use platform with training, top-down sponsorship, and clear launch processes.
The biggest thing that gets a team testing is to just do it. Stop designing the perfect experiment and get something simple live to take away the mystery.
Close every losing test with two questions: did it work for a granular segment, and is the idea worth further investment?
Scale experimentation with AI: use Cursor desktop/cloud agents for parallel builds and visual QA; orchestrate docs/analysis via Claude; automate cleanups and reporting.
Massey's first test removed navigation from UPS's shipping checkout flow and delivered $35 million in incremental revenue—proving e-commerce best practices apply even when customers think "this is just a tool, not e-commerce."
Democratize experimentation with a centralized platform and self-serve tooling; reset baselines regularly.
Prioritize like a pyramid: fix the widest-impact experiences first, then optimize down into smaller cohorts.
False negatives are more dangerous than false positives — they get institutionalized as "we tried that, it didn't work" and quietly kill good ideas for years.
.svg.avif){kind=logo}
A failed test can hold the real winner; contextual onboarding matched to user intent roughly doubled activation and became the default variant after the bundling experiment was rolled back.
Purge “anti-knowledge” by standardizing design, instituting cross-functional reviews, and only codifying learnings supported by repeatable data.
When you struggle to land a result, lead with the story of what the customer did, then bring the numbers.
Shift from MVP to MVT: list leap-of-faith assumptions and design minimum viable tests before you build.
Simplification has a limit. Removing too much can strip away the cues and context buyers actually need to decide.
Top-down buy-in shifts the conversation from "why test?" to "how do we test?": When leadership treats data as the tiebreaker, teams stop defending opinions and start building better experiments.
One centralized team of about 40 people tests every major change to Home Depot's $25B online business, serving 40–50 business teams with consistent hypothesis and analysis standards.
Win rate matters less than learnings per test — DoorDash ships company-wide experiment summaries (win or lose) that the CEO actively reads and responds to, creating cultural accountability around testing rigor.
