Guest Spotlight: Sebastien Polis

Sebastien Polis

Preply | Product

About Sebastien Polis

Sebastien Polis is Senior Experimentation Manager at Preply, where he leads the experimentation team, platform and culture behind nearly 1,000 tests a year. With a background in data science and experimentation at DAZN and Dojo, he focuses on rigorous measurement, iterative learning, global control groups and the challenges of testing across a two-sided learning marketplace.

No items found.

Takeaways

All Takeaways

In a marketplace, randomize the tutor and measure learner outcomes with a clustered analysis, and ship bug-free code because sample size is scarce

Go to S | E

A public Slack channel where execs question PMs about launches and kills keeps rigor and transparency high across 1,000 tests a year

Go to S | E

A surprising win from a simple change deserves a rerun; Preply unscaled a wording test and the lift disappeared

Go to S | E

Losing is the norm; about 80% of Preply's tests fail and 30 to 40% of each year's experiments are iterations of an earlier idea

Go to S | E

Preply ties OKRs to experimentation: teams can only claim results through experiments that use an approved target metric

Go to S | E

About Sebastien Polis

Subscribe to the podcast