GrowthBook 2.0 — Data Day!

This is the second day of feature announcements for our upcoming GrowthBook 2.0 release. Today, we’re focusing on changes related to data and statistics.

CUPED (Variance Reduction) for Faster Experimentation

Waiting to collect enough data can be a significant roadblock in increasing experimentation velocity. CUPED, a form of variance reduction, is one of the simplest and most effective ways to remove those roadblocks and get answers faster. For example, Microsoft estimates that using CUPED was equivalent to getting 20% or more of traffic for most of the metrics used by one of its product teams.

CUPED is now available in the Frequentist engine in GrowthBook 2.0. Read more about it on our engineering blog.

Improved SQL Editing Experience

Getting your metric queries exactly right can require a lot of switching between your database environment and GrowthBook. With the schema browser, you can now browse the database schemas directly from GrowthBook, making adding exposure and metric queries easier to build. Read more about this feature on our engineering blog: https://blog.growthbook.io/introducing-the-growthbook-schema-browser

Faster SQL and Simpler Configuration

GrowthBook 2.0 can execute SQL queries up to 2X faster than before. By introducing a new attribution model and applying other internal optimizations, the median query runtime is consistently better across test queries and up to twice as fast in certain configurations.

Furthermore, the new “Experiment Duration” attribution model makes it easier to analyze metrics from experiment exposure until the end of the experiment, rather than requiring arbitrarily large conversion windows. The following figure shows the two Attribution Models you can select from in GrowthBook 2.0.

You can read more about the faster SQL queries on our engineering blog.

Databricks Integration

GrowthBook now natively supports Databricks as a data source for experiment data. Simply add a new data source, choose the event tracker you use, then select “Databricks” and add your connection details. (This is the 14th data connector for GrowthBook — let us know if you have a request for any more!)