Warm Gun: Challenging Data Driven Design

by Luke Wroblewski November 22, 2013

In his presentation at the Warm Gun Conference in San Francisco CA, PJ McCormick shared how designers can work with data to help inform decisions. Here are my notes from his talk:

  • Data can teach you that you are not as smart as you think you are. Running tests and measuring real world use can humble you.
  • It can be frustrating to have data prove your design effort and instincts wrong.
  • The point of experiments isn't to be right. It's to get information so you can find the right answer eventually.
  • Quantitative data is the voice of your customers. This can help you make decisions.
  • "Helps you" is the keyword. Data makes decisions is dangerous.
  • Dig deeper before you overreact to data. Make sure your understand the information before you use it to make decisions.
  • Optimizing things without the full context of the experience can lead to lots of small optimizations at the expense of the whole.
  • Conversely, making decisions without any data leads to wishful designing that ignores real world use.
  • Don't test just to optimize, have a hypothesis, and use the data you get to inform your decisions.
  • Data requires you to layer investigative work on top of it to understand what it really means.
  • Information informs your long term thinking. It can't do it for you.