ConveyUX: Rethinking User Research

by March 6, 2013

In her Rethinking User Research and Usability Testing for the Social Web talk at Convey UX Dana Chisnell talked through some of the issues with current usability testing practices and how to adapt user research to today's social Web. Here are my notes from her talk:

  • The state of the art in usability testing is a generation old. Its time for a new generation of user research methods.
  • Usability testing isn't telling us what we need to know for the social Web It is awesome for testing the flow of a task and finding obstacles. But we also need to test relationships online.
  • People don't live in the real world doing one task with one device out of context. One computer, one task is not an effective way to test software. Social interactions.
  • Other people are part of the decision-making process that's part of using software.
  • More and more technology is about human to human interaction, not human computer interaction.

New Considerations

  • Usability methods and measures were probably always imperfect but they used to be enough.
  • The nature of being online is social. Anything that anyone does that effects someone else in some way. The nature of the Web is social -its all about sharing and interacting with others.
  • Scale is a game changer. It can provide more data, but be aware of biased samples.
  • Tasks aren't what you think. They are the centerpiece of usability testing, but we are often asking people to pretend. People don't come to technology with tasks in mind.
  • Their activities are changeable and ephemeral depending on context, people, and environments.
  • Satisfaction is correlated with task completion. Instead we should measure the degree of control and engagement people have with our services.
  • We need to know what keeps people coming back. If you took it away tomorrow, would they miss it and why?
  • Users continuously design your UI in real-time through work-arounds and their own techniques.
  • Social is all about context. Context defines relationships -where do our designs fit into relationships.
  • Look outside ourselves: social sciences, behavioral economics, linguistics, etc.
  • The most successful user research methods are hacked together using whatever techniques you need.


  • Review online profiles of people you are connected to, are they grouped in some way?
  • Ask stories of how people met.
  • Multi-user sessions: ask people to bring friends. See how they interact together.
  • Experience sampling. At random times, you ping research participants to see what is going on in the context they are in: through text or call.
  • This takes more time because interactions and relationships unfold over time. It requires strong research design and deep thinking.
  • We need to be clear about the questions being asked and why. What are the right questions?