Notable Quotes
"We have an annual revenue of $150 billion as the enterprise. And then for the online org that I'm working with we just surpassed $25 billion."
"Basically any major changes you want to go to the website homedepot.com, whether it being a content swap or a feature update or a model change in search or recommendation algorithm, it will be tested by my team."
"All the executives, leaders love to learn about the test. They really truly understand AB testing is the golden rule to understand incrementality, the real impact."
"We test because we don't know for sure... Are you gonna do something differently if the test result come out bad? If you're not, then let's not waste time."
"Right now we're still very much client-side testing, and we are developing server-side testing capabilities so everything moves faster, the end to end."
Takeaways
One centralized team of about 40 people tests every major change to Home Depot's $25B online business, serving 40–50 business teams with consistent hypothesis and analysis standards.
Scaling past low hundreds of experiments per year is a capabilities problem before it's an AI problem — Home Depot is moving from client-side to server-side testing so winners release quickly, end to end.
Kim's stakeholder filter: if you wouldn't do anything differently after a bad result, don't run the test.
