The challenge: multiple tools and high costs slowing innovation

Upstart’s experimentation and feature-flagging process had become complex and somewhat manual. The company was using multiple tools, including a third-party solution for feature flagging and two separate in-house solutions for experimentation, which created bottlenecks and slowed decision-making.

"We had to be familiar with these different tools and rely too heavily on our analytics team for data analysis rather than having software that could handle some of that for us. It was difficult not having experiment results available at all times, and it was slowing us down when rapid iteration was critical.”
— Diego Accame, Director of Engineering, Growth, Upstart

The delays had an additional impact as well: the potential for suboptimal results from the experiments that were in progress. Diego’s team decided it was time to look for a solution.

The search for a unified solution

Recognizing the need for a flexible, efficient platform, Upstart explored several alternatives:

• Investing in and expanding in-house solutions
• Buying third-party tools for feature flagging + experimentation
• Customizing free open-source solutions with additional investment

These alternatives did not fully address Upstart’s key requirements:

• Support for their diverse tech stacks: Flexibility was crucial given that different teams used different programming languages.
• Strong data privacy and security: As a financial services company, Upstart needed to securely host sensitive data in-house.
• Cost-effectiveness: The solution needed to be affordable, given Upstart’s current size and needs, and as it scaled its operations.
• Ease of use and autonomy: Engineers needed a tool that would allow them to design, launch, and analyze experiments according to their own specifications.

The GrowthBook ROI

Upstart considered building its own platform, but that would have required four engineers in the first year and at least two engineers thereafter. The engineers would be building tests instead of building product. GrowthBook eliminated the need to build an experimentation framework while dramatically accelerating experimentation results.

GrowthBook’s open-source, cost-effective model provided a ready-to-use solution that saved Upstart significant resources from day one. The platform empowered Upstart’s engineers to self-serve and, with real-time access to data, teams could make faster decisions, enhancing their ability to innovate at speed.

Furthermore, GrowthBook offered the ability to host all data in-house, ensuring full control over sensitive information, which was critical for Upstart’s compliance and security requirements.‍

"With the kinds of experiments we run and the sensitive data we handle, data security is paramount. The fact that GrowthBook offered us the ability to keep that data in-house was a key reason why we chose to work with them."
— Diego Accame, Director of Engineering, Growth, Upstart

Upstart’s decision to adopt GrowthBook was further reinforced by a successful proof of concept during a hack week, where the platform’s UI and capabilities were put to the test by a small team.

The migration process: updating complex systems

Though GrowthBook is the right solution, migrating isn’t without its challenges. Upstart has to consolidate three existing flagging and experimentation platforms into one new solution. This means cleaning up a large number of feature flags and experiments that use legacy systems, educating and onboarding GrowthBook to all teams with their codebases, and finally deprecating and removing the old tooling after the cleanup is complete.‍

"We had to rally our teams for this project and get our old tools cleaned up, but the process is significantly improving the quality of our systems and reinforcing best practices around experiments and feature-flagging hygiene."
— Diego Accame, Director of Engineering, Growth, Upstart

Results and benefits of adopting GrowthBook

Since adopting GrowthBook, Upstart has seen multiple key improvements:

• Consistency in Metrics: GrowthBook provided preloaded, standardized metrics that resolved the inconsistencies sometimes caused by human error.
• Faster Decision-Making: Engineers are able to assess experiment performance in real time, allowing for immediate, data-driven actions. This newfound autonomy has led to quicker identification of underperforming experiments and faster iterations.
• Improved Collaboration: Upstart’s experiment dashboards are easily accessible and shareable, promoting transparency and encouraging cross-team discussions.
• Reduced Cognitive Load: Using one tool instead of three streamlines maintenance while reducing cognitive load for engineers writing experiments.
• Cost Efficiency: GrowthBook’s open-source model provided a budget-friendly alternative to other solutions, helping Upstart stay within their financial targets while improving their experimentation processes.
• Reduced Technical Debt: The migration to GrowthBook led to the cleanup of old feature flags and experiments, improving overall system hygiene and maintainability.‍

"We've been able to see underperforming experiments quickly and react immediately, empowering our engineers to make decisions independently."
— Diego Accame, Director of Engineering, Growth, Upstart

Conclusion

Upstart's journey with GrowthBook demonstrates that the right tool doesn’t just solve technical problems—it transforms how teams work and innovate. By switching from a fragmented system that included third-party and in-house tools to GrowthBook, Upstart saved time and resources while improving their ability to make data-driven decisions. Upstart is now able to launch experiments in hours when it had taken days in the past.

GrowthBook has empowered Upstart's engineering teams to innovate faster, reduce dependency on manual analytics processes, and ensure data privacy with its on-premises solution.‍‍

"GrowthBook has changed the way we think about experiments. It allowed us to uplevel our code, speed up decision-making, and focus on what we do best—building a world-class AI lending marketplace."
— Diego Accame, Director of Engineering, Growth, Upstart