Notable Quotes
"Revenue is an outcome. What we really want to look at are the metrics that lead to the outcome of revenue."
"Not every page on the site is responsible for conversion. If we're doing a test on a product detail page, the cart is really responsible for conversion at that point. The PDP is responsible for getting the user to add the product to cart."
"A/B testing is going to tell you what happened. It's going to show you the data and the numbers, but user testing is really what we need for some of that nuanced why: why did they make that choice? Why didn't they click that button?"
"The team altered that to match a search term rather than forcing the customer into a product and risking getting that wrong. And we actually saw double-digit increases in feature usage and also add-to-cart from those searches."
"The first thing starts at inception of the test: really asking the product manager, what problem are you solving for the customer? Originally we looked at creating a hypothesis off the bat, but even that was too far in the funnel."
Takeaways
Dorothy is building personas into agents so browsers, not just buyers, can be tested for a directional read before an experiment ships.
Every test starts with one question to the product manager: what problem are you solving for the customer, not what is your hypothesis.
A/B testing tells you what happened. MilliporeSigma's user testing ambassador panel supplies the why behind the click.
