Notable Quotes
"I believe there was somewhere around 4 million-ish people and only 106 people clicked on the button. And I will never forget that 106 because it was such a huge, stark, obvious evidence that this is not something people wanted."
"So we did what I call dry testing, which is essentially testing a feature without that feature actually being built."
"It ended up saving the business, I would imagine, upwards of multimillion dollars. It's not just about building the capability digitally, but now you have to hire folks that can video chat. You have to train them."
"The idea of not calling them wins and losses, but savings and gains. If a test won, you gained something. But if a test lost, really we saved ourselves."
"One of the biggest unlocks is to just do it. Getting out of your head of having the perfect test. Just get something live."
Takeaways
The biggest thing that gets a team testing is to just do it. Stop designing the perfect experiment and get something simple live to take away the mystery.
Getting a stuck team unstuck starts with data and a workshop. A Disney team went from "we don't know where to start" to 110 scored, prioritized test ideas, using Contentsquare heatmaps to diagnose low engagement first.
Slow experimentation is often a tooling problem, not a methodology problem. One program's six month test cycle came from rebuilding every page instead of overlaying changes the way the tool intended.
