Oda Built a Faster, Smarter Experimentation Practice

Executive Summary

Before GrowthBook, Oda’s experimentation setup was homegrown, hard-coded, and tedious enough that experiments felt like major projects. The team wanted a better way to test ideas, measure impact, and improve product decisions without fighting the tooling every time.

GrowthBook fit because it worked on top of Oda’s existing stack. It sat on top of Snowflake, reused the data warehouse and metrics the team had already built, and let Oda keep using experiment data in Looker and Jupyter for deeper analysis. That made it easier to adopt and easier to trust.

Today, Oda uses GrowthBook to improve recommendation models, support controlled rollouts, and make better calls on changes that affect both customer experience and profitability. One recommendation improvement drove 16-17% usage growth. A loyalty program was iterated for months through A/B testing before full launch, helping the team make sure it was profitable before rolling it out broadly. Most importantly, experimentation is now part of how Oda builds products.

From hard-coded tests to a real experimentation workflow

Oda started with a homegrown setup built on PostgreSQL and hard-coded feature flags. It worked, but it made experimentation slow and manual. The team described the post-experiment analysis process as tedious, and experiments themselves as “big things” rather than a normal part of development.

That changed when the team found GrowthBook in 2021 and tested it on top of the stack they already had. The appeal was straightforward: it worked with their existing data warehouse, it didn’t force them into a black box, and it felt plug-and-play compared to what they had before.

Oda did look at alternatives, including Optimizely. One reason GrowthBook stood out was data ownership. Instead of pushing more event data into a vendor system just to define new metrics, Oda could query the data it already had. For a team that had already invested heavily in Snowflake, dbt, Snowplow, and BI workflows, that mattered.

“I recall GrowthBook being a massive way to harvest the investment we made when building the whole data pipe. It was so plug and play that putting it on top of Snowflake was really amazing.”
— Andreas Fischer, Product Manager, Shop Team, Oda

Why the Bayesian approach worked for Oda

The technical fit mattered. So did the stats.

Oda said the Bayesian approach fits well with how its teams actually build products. Instead of doing a lot of rigid upfront planning around sample sizes and event thresholds, they could follow experiments as evidence accumulated and use that to guide decisions in a more natural way.

There was still a learning curve. The team said one early challenge was getting better at interpreting uncertainty. But once that maturity developed, experimentation became easier to use as part of regular product work instead of something that sat off to the side.

That mattered because Oda does not rely heavily on user testing before shipping. The team uses experimentation to get signals they might not otherwise get until much later, or at much higher cost. In practice, that means GrowthBook helps the team move faster without flying blind.

“Just knowing that we will get earlier signals from GrowthBook helps a lot in daring to do stuff.”
— Fredrik Jørgensen, Head of Insight, Retail Platform, Oda

Recommendation models got easier to improve

As an online grocery company, Oda needs to show the right products to the right customers. Some of that is repurchase behavior, which is relatively straightforward because people buy the same staples repeatedly. Some of it is product discovery, which is harder because Oda is trying to recommend products a customer has not bought before.

That second problem is where live experimentation matters most. Offline evaluation can only tell you so much when you are changing what people see and influencing what they buy. Oda uses GrowthBook to test recommendation changes in production, measure what actually improves behavior, and iterate from there.

That workflow produced one of the strongest hard number outcomes. Oda said a refinement to its repurchase algorithm drove 16-17% growth in usage, and described the result as conclusive and unusually strong for a single model iteration.

“Experimenting a lot with personalization has been very valuable to guide the development of a recommendation model.”
— Lars Mushom, Data Scientist, Shop Team, Oda

GrowthBook helped Oda measure tradeoffs, not just wins

Not every change is about a simple lift in one metric.

Oda described situations where the team knew a change would reduce customer friction, but also knew it might come with a cost.

“The real question was not ‘does this improve the experience? It was ‘how much does it cost, and is the tradeoff worth it?’ Experimentation with GrowthBook gave the team a way to answer that with data instead of intuition alone.”
— Fredrik Jørgensen, Head of Insight, Retail Platform, Oda

That’s an important part of this story. Oda is not only using experimentation to find big winners. It is using it to make better product decisions when customer experience, profitability, and operational realities all pull in different directions. One strong example of this is their loyalty program.

Instead of launching it to everyone and hoping it worked, the team launched it as an A/B test and kept iterating for months before full rollout. That gave Oda time to understand the mechanics, improve the experience, and make sure the program was profitable before scaling it across the customer base.

That matters because discount-based programs are easy to get wrong. They can drive activity while quietly crushing margins. Oda used experimentation to avoid that trap.

“We ended up making money by giving those discounts, which is really a hard challenge. We were able to thread ourselves toward it, and that was a lot due to A/B testing and GrowthBook.”
— Fredrik Jørgensen, Head of Insight, Retail Platform, Oda

The biggest change was cultural

The headline outcome is not just one experiment. It is how Oda works now.

Oda is “always experimenting,” and experimentation has become a natural part of product development. With 400+ experiments, they’ve driven a broader change so that now experimentation is something engineers, PMs, and growth managers do consistently and independently.

Oda currently has around 50 active experiments, with up to 10 larger experiments that get deeper analysis. That mix is a great example of a portfolio of experiments, some that are big swings and others that are smaller optimizations. 

“Our culture has totally changed. Experimentation went from being a ‘big, hard thing,’ to just a standard part of product development.”
— Fredrik Jørgensen, Head of Insight, Retail Platform, Oda

What Oda is exploring next

Oda is still pushing forward.

The team points to several areas they are excited about next, including GrowthBook’s MCP capabilities, multi-armed bandits, and CUPED-style analysis to make experiments faster and more reliable. That fits the broader pattern in this story: Oda sees experimentation as core product infrastructure, not just a reporting layer.