
Notable Quotes
"I always say that being a product designer is a master class in removing your ego from the experience. You cannot get too attached because users will always find ways to use your product differently than you expected."
"We do a lot of experimenting by either releasing things internally or releasing them in small cohorts to see whether or not the experiments win or lose. It's our way to make sure that we can action against our business metrics and we don't disrupt our core revenue or order metrics, but we can still see if something's going to work or it's not."
"If you can talk to a user about what they saw and why they made a decision, it can really help clarify a lot of your quantitative metrics. The marriage of qual and quant is a really great way to get a holistic picture of what happened."
"If you ask a user what they thought of this button or why they didn't click it, and they say, I didn't even notice it, that's a really great data point, because then this is either not coming at the right part of the process or it's not visible enough to somebody who's only half looking at their screen."
"At some point you need to de-risk. It's either in that fast follow or it's upstream in the process. But what is your intentional path through making sure that you're delivering positive features? It should be intentional. You should be thinking about that as part of the product development process from end to end."
Takeaways

Both upstream de-risking and fast iteration work. The right choice depends on how quickly engineering can ship, and it should be made on purpose.

Pairing quantitative results with five to ten user interviews after a test explains why it underperformed, including whether users even saw the change.

AI lets Grubhub's design teams put clickable prototypes in front of users early, turning each iteration into a micro experiment.

