B2B product discovery: what MilliporeSigma learned rebuilding on-site search

Most experimentation stories start with a bold redesign. This one starts with a search box.

Dorothy Crepin is the Senior A/B Testing Analyst at MilliporeSigma, the name Merck KGaA's life sciences business goes by in North America. She joined in 2022, a few months after the A/B testing program started, and she describes her job the way a dispatcher would: air traffic coordinator for the testing program, moving experiments from inception through execution to analysis across product managers, designers, engineers and analysts.

Three audiences, one catalog

Personas get treated as a marketing exercise at a lot of companies. At MilliporeSigma they decide whether the site works at all.

The traffic splits three ways. Scientists arrive looking for a specific instrument or material for an experiment. Procurement buyers arrive with a list somebody else wrote and no interest in alternatives. And a large education population arrives from universities, learning rather than buying, with no purchasing power at all.

Layer a regulated industry on top of that. What MilliporeSigma can sell changes country by country, and so does the way the company presents its own name. The same catalog has to behave differently depending on who is looking at it and where they are standing.

The change that beat the obvious answer

Dorothy's team noticed that customers were reaching the site through Google rather than using the site's own search, and that the on-site search was underused once they arrived.

The original type ahead did what most e-commerce type ahead does. Start typing, and it recommends a product that matches the query. The team changed it to suggest a search term instead, and stopped forcing the customer into a single product the system might have guessed wrong.

The result was a double digit increase in feature usage and a double digit increase in add to cart from those searches.

What makes it a good experimentation story is that the winning variant is the less commercial one. Pushing the product is the instinct. Letting the customer describe what they are looking for is what worked, and it opened a question the team is still pulling on: how quickly to show options, how literally to take a query, and whether a wider set of products now gets discovered and bought.

Revenue is an outcome

Dorothy traces a change in how her team measures back to a webinar she watched about 18 months ago.

Everyone focuses on revenue, and stakeholders are measured on it, so it is never going away. But revenue is an outcome. The useful question is which metric leads to it, page by page.

Her example is a product detail page. The cart is what converts. The product detail page's job is getting the product into the cart, and before that it is product discovery. Measuring a PDP test on revenue measures somebody else's work.

That framing also shapes how her team sequences tests. When four or five experiments are ready at once and all of them touch the same area of the site, the questions become whether they can be made mutually exclusive, whether there is enough audience to split, and whether the change is big enough to measure at all.

Where A/B testing stops and user testing starts

A/B testing tells you what happened. It does not tell you why somebody did not click the button.

MilliporeSigma's user testing group built an ambassador panel for exactly that gap: people with real domain expertise rather than the general public, which matters when your customers are scientists. Dorothy calls the combination of the two a powerful tool the company is still working on unlocking, which is a more honest description than most programs give.

Start with the problem, not the hypothesis

When a product manager arrives certain that a feature will win, Dorothy does not start with the hypothesis. She has found that a hypothesis is already too far down the funnel.

The first question is what problem are you solving for the customer, followed by what the solution looks like when it is finished. Then, before the test is built: is this customer problem grounded in data, or is it a gut feeling? Her team has data scientists building dashboards and self service tools precisely so the answer can be the former, and she is candid that a gut feeling is not disqualifying. It just should not arrive alone.

What Dorothy is watching next

Two things.

The first is agentic testing. MilliporeSigma has relatively few people going through the purchase process and a great many browsing. Dorothy wants to build personas into agents to get a directional preview of an experiment and to test the browsing majority in a way that still connects to the outcome.

The second is how tests get evaluated. Her team is looking at multiple models and methodologies rather than one Bayesian approach across the whole program, and she is seeing vendors move the same way. She also expects analysis to go beyond primary and secondary metrics, to something that surfaces the one standard deviation anomalies nobody thought to look for.

For a program aiming at twelve experiments a quarter, that is a lot of ambition. It is also what happens when leadership treats testing as part of the product development lifecycle rather than a check at the end.

🎧 Listen to the full episode →