"So we make customers that look like our customers, but there's no real data being used."
"Testing gets watered down as a concept, as in let's try out something, not necessarily what often what I think and you think experimentation testing would be, which is a control group to this new variant."
"We can actually leverage web behaviors to see where people are getting looped and stuck, and how many times they revisit the same section over and over again."
"What I think is important for an experimentation mindset is that there you can't get punished for making mistakes. You just need to be able to learn from those mistakes."
"They got over-touched. They realized they had value and started to look for people who wanted to pay for that value."
Takeaways
An experimentation mindset requires that people can't get punished for mistakes; guardrails plus a safe playground beat running the same test forever.
A center of excellence that shares wins and losses turns tribal knowledge into shared knowledge and stops "we tried that years ago" from killing retests.
Testing gets watered down when "let's try something" replaces a control group; a little pre-planning gets far more out of every experiment.
