"Your one test might not have a huge amount of impact, but when you look back over a year, the 100 or so tests you run might. The growth will have compounded over time."
"It had absolutely zero impact, zero impact on user behavior. It didn't increase purchases. It didn't decrease purchases."
"Everyone new to this has to go through this kind of ego death, where their brilliant idea actually has zero impact whatsoever."
"If you understand the problems you're trying to solve, naturally you'll come up with multiple hypotheses for each problem. One problem can spring a nice tree of ideas."
"I guess you could call it democratizing experimentation, giving the power to the people."
Takeaways
Scaling experimentation from 0.3 to 2.8 tests per month is less about education and more about habit change, shared learnings, and giving non specialists the tools to launch their own experiments.
Problem mapping on a 2x2 matrix of evidence versus impact turns customer research into a prioritized experiment roadmap, and one validated problem can spring a whole tree of testable ideas.
Moving new experimenters from solution space to problem space thinking raises win rates and produces learnings the whole organization can use.
