Google Optimize Compared to Growth Book
Google Optimize is the largest experimentation platform currently since Google made it free in 2017. Google Optimize is the free testing tier, and a paid plan called Google Optimize 360. Both these tools offer a visual front side editor for creating tests, and integrates with Google Analytics for the data. Here we’ll highlight some of the major differences between Google Optimize, Optimize 360, and Growth Book.
Google Optimize is free, which is great. However, for that you’re going to face some limits. Users of Google Optimize can only have up to 5 tests at a time, and can only have up to 3 goals. You will also face limits on how many users you can analyze before you get sampling. You also cannot use Google Analytics Segments on the results without upgrading to the 360 version.
Growth Book wants you to test as much as you can, so there are no limits on the number of tests, number of metrics, or goals.
Google Optimize is free, but Google Optimize 360 starts at around $150,000 yearly. Optimize 360 pricing is fixed and gives you a certain amount of ‘hits’ to the AB tests. This means that the more you test, it’s possible to need more ‘hits’ to accommodate.
Growth Book believes that you should test as often as you can, and you only pay for seat used, not for traffic or MTUs. Furthermore, our pricing is significantly cheaper than Google Optimize 360, and we also offer a completely free version if you want to self host.
Google Optimize and Optimize 360 have a very slick front end AB test editor, and this is great for copy changes. Image changes are not supported through their front end editor. However, if you want to integrate deeply with your code, you’ll have a much harder time. Highly impactful tests are seldom the result of simple UI or copy changes. Impactful changes are usually larger and require code changes to implement, which is more difficult with tools like Optimize. If your website uses ReactJS or other modern client side rendering libraries Google Optimize may either break the page or cause flickering even if implemented correctly.
Growth Book encourages you to have a deep integration with your code. The variety of tests you can run and the chances of having successful tests are greatly increased if you consider the full range of what you can test, rather than just copy. We were built by developers for developers, so our SDK and APIs are simple and easy to use. We natively support React, and other modern tech stacks. If you would rather run your tests on the front end, we also have a front end editor that doesn't for less technical folks - or to pass on to the marketing team without getting engineering involved in every test.
Google Optimize integrates with Google Analytics, which you are probably using anyway. This is great, as it means that you can get up and running quickly. However you don’t have to be a data analyst to have run into the limits of what GA can do- this is the reason products like Amplitude and Mixpanel exist. When you want to dig deeper with your analytics, or AB tests, Google Analytics will frustrate you.
Growth Book also integrates with Google Analytics. When you outgrow what GA can do, we support a multitude of other data sources, from Mixpanel to Amplitude to Redshift to Snowflake.
Google Optimize acknowledges that creating a growth culture is very important (they even have a guide to it), but their tool primarily focuses on short term results. You can go back to see old tests and their results, but you won't see what it looked like, nor any context about what was tested and why. Furthermore there is no way to abstract out learnings, gather insights, or tag experiments for easy onboarding.
Growth Book is designed to help you be successful with your A/B testing program. To this end we are designed to help companies integrate an experimentation culture. Namely this comes from the ability to have discussions around any aspect of your experiment, easily share results of an experiment, summarize the conclusions and results, and create insights with larger themes and the evidence of them. Experiments are hypothesis driven, and hierarchical, so if one test sparked another idea to test, you can track that chain of thought. Particularly important is capturing the discussion and context around why an experiment was created, and what happened next so that you are capturing this tribal knowledge, which accelerates onboarding and de-risking departures.
Google isn't specific about where they store their data, and provides no way to delete data if required by a GDPR request. If you are in the EU, this might put you afoul of the EU Data Protection Directive.
Growth Book doesn’t store your users data, just the aggregated results, so there are no compliance problems.
Google Optimize is a great tool if you are just getting started AB testing. However it doesn’t encourage companies to test beyond simple tests like copy, doesn't encourage a growth culture, and limits the number of tests you can run. A/B tests usually have a low success rate, so you should be running lots of tests to increase your chances of successful results. Companies that grow their sophistication with AB testing even slightly will quickly hit limits and either be faced with limiting their tests or paying a lot for Optimize 360, and at that level, Growth Book is much cheaper.
|Google Optimize||Google Optimize 360||Growth Book|
|AB Testing||yes, limit 5||yes, unlimited||yes, unlimited|
|Data Sources||Google Analytics||Google Analytics||Any|
|AB test focus||front end||front end||deep integration & front end|
|Review past tests||limited||limited||yes|
|Capture discussions & context||no||no||yes|