To help me work through some recent thoughts I’ve had about Design Research, I asked Steve Portigal -founder of Portigal Consulting and all around bright guy- to talk about context within digital products and the connection between ethnographic research and design. Part one of our conversation follows.
Luke Wroblewski Just to set some context here, most of my experiences with design research have been for Web or Desktop software design. For a long time, this meant usability testing. Over the past few years, however, I've been part of an increasing number of ethnographic studies that take product testing out of the lab and into people's homes.
The researchers I work with feel they are more successful engaging customers outside the sterile confines of a usability lab and in the context of their natural environment. This provides them with an opportunity to discover unmet needs that ultimately become business goals. But somehow I feel I'm missing something.
In my experience an ethnographic study for a Web site usually amounts to watching someone work on their personal computer set-up and Internet connection. All the activity is on the screen. Most of the context is online. So shouldn't "real" ethnographic research for these types of products take place online? What is the biggest advantage of going to their homes? Seeing their environment? Their distractions? Their use of offline data and artifacts? I can see it being useful to gain insights into how offline processes are used in conjunction with online process like shipping an item you sold on eBay. But when the product, the community, and all the interactions happen online? Isn't a huge amount of the context digital?
Steve Portigal I'm going to sidestep any discussion of what "real ethnographic research" means because we're going to get into that later, I think where I will once again sidestep it. Here's a few thoughts on what you've raised - and I think you've hit on some of the core advantages already but I'll expand (admittedly it may start to bleed together)
The Personal Computer
People have their own computers, with their own browsers, plug-ins, bookmarks, history, and whatever else. The tools they use are individually configured, personal, and varied. I've handed someone a PC and asked them to show me how they would accomplish some task. And their first response would be "Well, I'd look in my bookmarks for [site] and then...." Oops. Now whatever we were asking them to do was extra-challenging and our observations were going to be skewed. Heck, handing them our testing-lab mouse could immediately send them off into adapting-to-our-tools. In that example, I was really surprised by how hampered people were by what they did or didn't have with this machine versus their own machine. Even down to what default screen we started them off with. We set it to MSN.com before they came in, and of course some people decided to start the task we gave them by using MSN, absolutely not what they would normally do. Heck, I use the Google toolbar - put me down at someone else's machine and I'm really at a disadvantage (I'm sure we could come up with a million different examples of this sort of thing).
Sometimes there is data (or a trailhead for a line of inquiry) from their desktop picture. There's richness there, it's great to have the chance to ask about it.
Where The Action Is
Are we always sure that all the activity is on the computer? What if, in order to complete a task I look in a book, look at a post-it note on the wall, make a phone call, review a printout or some other offline piece of information. I'm not talking about shipping, a larger offline interaction; I'm talking about micro-interactions that are interleaved with the key task, on the computer.
I really don't agree that all the context is digital. Unless someone is fully jacked into the Matrix, the key behavior we are looking at may be digital but the context is meatspace. Where our customers really live.
One important way to establish rapport in any research process is to be on someone else's turf. It's important philosophically. We go to them. We're in their space. We're going to learn from the person right from the moment we pull up in front of their house. Even if we spend 98% of the time in front of the computer, we're going to see the room filled with 20 dead PCs and understand more specifically what their off-the-cuff comment "I buy a lot of gadgets" means. There's an important aspect of serendipity to being in their world - we allow the unplanned to happen. The phone call (as you say, the distractions), or the roommate dropping by to share her version of some success or failure she's had in solving the problem we're curious about.
Luke Wroblewski No argument from me, there. Native environment certainly influences behavior in all the ways you've described: personalization (browser bookmarks and home pages, hardware set-up), integration of environment (notes, phone calls, roommates), and rapport (being in their space). I guess what I am looking for is the equivalent of the grocery store in the infamous IDEO "shopping cart" video.
For those unfamiliar with the piece, it's a documentary of the IDEO team "hitting the streets" to learn how they can improve the standard shopping cart. They observe how people use carts in context: within actual grocery stores. Seeing how people interact with the products, the people, and the spaces inside the store gives them a clear set of considerations for improving things.
So how do you get into that context online? Web applications are becoming increasingly immersive and social. To use the de facto example of a social Web application, let's look at the ecosystem of flickr. The contacts, groups, pools, sets, streams, and more within flickr create a lot of different contexts that shape people's behavior. How do we get at the digital versions of personalization, integration of environment, and rapport that we know are important in the offline world.
Steve Portigal I'm glad you refer to the IDEO example as "infamous" - because it's not a real example. It's a made-for-TV special that is filled with fakery. In one shot, a woman is walking through a grocery store supposedly documenting the environment with a digital camera. But she's an IDEO staffer, not an actor, so her body language as she pretends that the camera is not watching her pivot right and left snapping away carefree outs her (and indeed the whole process) as manufactured.
The camera did not document IDEO doing what they do, IDEO agreed to stage an event specifically to be documented. They created (and the producers edited) an idealized process that none of our work will ever live up to, because it can't, because it's real and constrained and challenged in ways that real projects always are.
As far as the cart goes, the grocery cart is part of the shopping activity. People are involved in the activity when they make a shopping list, when they schedule the trip to the store, in the store and packing up the car (of course), and then back home when they are putting groceries away and looking at what they've got in the context of their own home and kitchen (from storage space to emotional reactions of other family members to purchases).
Same with photography, right? There's a moment that is experienced when the shutter is opened, there's a decision to select and upload the picture, there's a process of tagging or whatever, and then there's interesting things that happen afterward...going to a party and having someone ask you about a recent trip or a stranger opening up a communication with the photographer of an image they have a connection with.
But diving into flickr as an example. You've got a lot of methods that you can pull in (and I'm certainly no methods junkie - but let me take a shot) and integrate, depending on what usage your are interested in (and what you want to get out of that)
- Log every usage automatically
- Ask the person to log every usage that fits certain criteria ("beeper study" or "diary study") and then go back and revisit those examples and ask about them
- Sit with the person and ask for a guided tour of how they use some aspect of flickr
- Sit with the person and ask them to show you various features - including their impressions of things they may not use regularly, or ever
- Sit with the person and ask about flickr ("show me how you would do X....") but probe on analogous examples that they could show you (Q: Show me your contacts on flickr A: They're pretty basic, like this Q: Do you use other sites or programs where you have a list of contacts? A: Yeah, I'm way into LinkedIn Q: Okay let's take a look at that, then....)
- Show the person some simplified or storyboarded idealized version of flickr, or of a generic application, and have them walk you through how they would solve some kind of problem
I'm obviously going over a lot of ground you are familiar with above, but maybe at this point you could clarify what you're pushing at in terms of getting into these digital contexts, and I'll try and be more relevant in my response!
Read part two of this conversation on Steve’s site.