An Event Apart: Designing in the Space Between Devices

by Luke Wroblewski May 11, 2014

In his Mind the Gap: Designing in the Space Between Devices talk at An Event Apart in San Diego CA 2014, Josh Clark talked about the opportunities and challenges of designing interactions between devices. Here are my notes from his talk:

  • We're starting to get a better handle on how to get our services and information on our screens. But we need to also consider the gaps between our devices.
  • 15 years ago, Palm Pilots made it really easy to beam contact information between devices. This has gotten harder today. We have multiple devices and multiple standards to contend with.
  • In the UK, the people switch devices 21 times per hour (TV and smartphone). 90% of people move between devices and use them simultaneously.
  • People hack connections between devices: search, email, and text. The systems we have aren't ready to cross the gap between devices. Google search and email are the connectors between devices.
  • We are still using "remote control" style interactions to get things between devices. We still beam things to screens today.
  • Synching content has gotten much better for static content. But our challenge is to create interactions that allow us to shift behaviors and content across devices.
  • For sequenced tasks, we can use inputs like cameras to bring up content on different screens. Start on one device and move to the next.
  • For simultaneous tasks, we can use technologies like Web sockets and Web RTC to create browser to browser interactions that happen across devices in real time.
  • Bluetooth LE provides identification of devices so they can interact with objects.
  • It's not enough to just share content across devices, we want to share actions as well. How do we sync verbs not just nouns.
  • You can use Web audio to beam a unique ultrasound signal form one device to another. That can trigger actions or even authentication for things like login.
  • Screens encumber and constrain us. They create opportunity but anchor and distract us from the real world. When we're interacting with screens, we're not interacting with people.
  • Don't design for screens, design for people.
  • Make use of physical real-world interactions. Examples: DrumPants allow you to tap your pants to make sounds.
  • The physical nature of interfaces can make them feel more natural. Proximity has a lot of potential as an interaction model.
  • Cross-device interactions are not challenges of technology today. Our devices have all the technology we need. They're challenges of imagination. We have the tools to make this happen today but need to apply ourselves to this kind of design.
  • We have so much technology, we can create new magic on a daily basis.

Digital and Physical

  • Digital interfaces have been becoming more physical over the past few years as we've created more mobile computers.
  • At the same time, physical objects are become digital. It is increasingly easy to add connectivity and sensors to everyday objects. We've got connected baby outfits, wigs, and even egg trays.
  • Our job as designers to help people be as lazy as possible.
  • Anything that can be connected will be connected. Everyday objects are now digital gadgets. Physical things have digital representations.
  • Social machines: what does it mean when devices are participating in our digital lives?
  • Software makes hardware scale. An endless variety of content can come from devices connected to people.
  • LG has created a chat interface for their appliances. You can chat with them to collect information and start/stop tasks.
  • Connected devices won't always say nice things. They can be hacked like other devices.
  • Cars, appliances, and more can have security and privacy issues we need to be mindful of as well.
  • Software is ideology, embedded with values. The genie is out of the bottle, the systems are out there already. So it is up to us to do the right thing.
  • We're already switching between devices. Plan for it in your designs.
  • Don't just sync content, sync actions and processes.
  • Peer to peer technologies (even in the browser) can make connections between devices.
  • Think about how you can move interactions off of screens and onto sensors.
  • As we close the gap between physical and digital experiences, how do we create avatars for objects.