Optimizely Compared to Growth Book
Optimizely is one of the largest experimentation platforms, and also one of the most expensive. Built primarily for marketing teams, optimizely used to be the default for AB testing before Google Optimize came along. Both Growth Book and Optimizely offer solid AB testing statistics and results. Here we’ll highlight some of the major differences between Optimizely and Growth Book, and why we think Growth Book is the smarter choice.
Optimizely does not disclose their exact pricing, but billing is done per user per test, so the more you test, the more you pay. For example, a product getting 200,000 visitors a month can expect quotes of around $36,000 a year for a normal cadence of 1 test a week.
Growth Book believes that you should test as often as you can, and you only pay per seat, not based on traffic. We also have an entirely free open source version if you want to host it yourself. This makes Growth Book significantly cheaper than Optimizely.
Optimizely requires you to use their customer tracking, which presents a few problems. One of them is that you now have yet another place where personal information on your users is stored, which presents compliance issues, particularly with GDPR, HIPAA, COPPA, etc. Another is that you cannot tie into existing metrics and calculations you might already have on your users (such as custom LTV calculations, or engagement metrics). Finally, many ad blockers and do not track browser extensions will block Optimizely from working completely.
Growth Book works with your existing data sources, whatever they are. In this way we are compliant with most regulations. Our AB tests are tightly integrated with your code, which makes them immune to ad block problems.
Optimizely primarily focuses on short term tests. You can go back to see old tests and their results, but if your site has changed significantly, seeing exactly what the test looked like, or what the variations were specifically is not possible. Furthermore there is no way to abstract out learnings.
Growth Book is designed to help you be successful with your AB testing program. To this end we are designed to help companies integrate an experimentation culture. Namely this comes from the ability to have discussions around any aspect of your experiment, easily share results of an experiment, summarize the conclusions and results, and create insights with larger themes and the evidence of them. Experiments are hypothesis driven, and hierarchical, so if one test sparked another idea to test, you can track that chain of thought. Particularly important is capturing the discussion and context around why an experiment was created, and what happened next so that you are capturing this tribal knowledge, which accelerates onboarding and de-risking departures.
Growth Book encourages deeply integrated tests which do not expose the tests cases to the end users.
Optimizely is a good A/B testing tool built for marketing teams. Once you want to test beyond these simple tests, you'll hit limits with what Optimizely can do. Futhermore Optimizely charges more the more you test, which is counter what you need to do to ensure a successful A/B testing program. A/B tests usually have a low success rate, so you should be running lots of tests to increase your chances of successful results. Optimizely is not built to encourage a growth culture, it's tools make it hard to revisit context around old tests, or leverage insights for the future. Growth Book a cheaper and better option.
|AB Testing||unlimited, but you pay more||unlimited|
|Data Sources||Optimizely||Any, (including optimizely)|
|AB test focus||front end (back end integraion avalialbe)||deep integration & front end|
|Review past tests||limited||yes|
|Capture discussions & context||no||yes|