Web App Summit: A Netflix Case Study

by Luke Wroblewski January 22, 2007

At the UIE Web App Summit in Monterey, Sean Kane outlined the Netflix Web development process and culture through an overview of how many innovative features became part of the Netflix user experience online:

Netflix’s mission is to connect people to movies they'll love. The company strives to deliver magical, unexpected moments for their customers and the company culture is a huge part of what makes that possible. Netflix has 5.7 million members, over 70,000 titles, more than 95% of all their titles are rented each quarter and the team has regular two-week development pushes. The Web experience team strives for easy, personalized, innovative, and cinematic experiences.

  • Because of the amount of titles available on Netfilx, personalization is critical. The fact the site evolves to match your tastes, makes the quantity of films manageable.
  • Intuition is only a start. Netfilx is willing to explore all ideas but many of them will not be successful. Many of the things they try do not work
  • In order to prove success, you need to define the right metrics. Netfilx tracks total stars added, number of interactions, time to task completion, and more.
  • Find gems in tests. Don’t throw an idea out if part of it does not work. Evaluate against each of your success metrics.
  • Don't become emotionally involved in your designs. Be willing to test & verify.
  • Once you have qualitative & quantitative data you can rally around solutions

If a product goes untested...

  • You become very reactive
  • You become very defensive
  • Quick fixes pushed out are not tested, resulting in a vicious cycle
  • Seasonal obfuscation may cloud your understanding of the data
  • No real-time control group if comparing pre- and post-launch data