In his Mind the Gap: Designing in the Space Between Devices talk at An Event Apart in Chicago IL 2014, Josh Clark talked about the opportunities and challenges of designing interactions between devices. Here are my notes from his talk:
- We're starting to get a better handle on how to get our services and information on our screens. But we need to also consider the gaps between our devices.
- 15 years ago, Palm Pilots made it really easy to beam contact information between devices. This has gotten harder today. We have multiple devices and multiple standards to contend with.
- In the UK, the people switch devices 21 times per hour (TV and smartphone). 90% of people move between devices and use them simultaneously.
- People hack connections between devices: search, email, and text. The systems we have aren't ready to cross the gap between devices.
- We are still using "remote control" style interactions to get things between devices. We still beam things to screens today.
- Synching content has gotten much better for static content. But our challenge is to create interactions that allow us to shift behaviors and content across devices.
- Apple recently announced Handoff that always you to move tasks seamlessly between devices: start an email on your phone, continue you it instantly on your laptop.
- For sequenced tasks, we can use inputs like cameras to bring up content on different screens.
- For simultaneous tasks, we can use technologies like Web sockets and Web RTC to create browser to browser interactions that happen across devices in real time.
- Its not enough to just share content across devices, not just content. How do we sync verbs not just nouns?
- Bluetooth LE provides identification of devices so they can interact with objects.
- How can we have computers literally "talk" to each other? Use can use Web audio to beam a unique ultrasound signal form one device to another. That can trigger actions or even authentication.
- Screens encumber and constrain us. They create opportunity but anchor and distract us from the real world.
- Don't design for screens, design for people.
- Make use of physical real-world interactions. Examples: DrumPants allow you to tap your pants to make sounds. Our goal isn't to make things frivilous or silly but designing toys allow us to explore and learn.
- The physical nature of interfaces can make them feel more natural. Proximity has a lot of potential as an interaction model.
- Cross-device interactions are not challenges of technology today. They're challenges of imagination. We have the tools to make this happen today but need to apply ourselves to this kind of design.
- We have so much technology, we can create new magic on a daily basis.
Digital and Physical
- Digital interfaces have been becoming more physical over the past few years as we've created more mobile computers.
- At the same time, physical objects are become digital. It is increasingly easy to add connectivity and sensors to everyday objects. Anything can become a source of data and a controller of data.
- Our job as designers to help people be as lazy as possible.
- Anything that can be connected will be connected. Everyday objects are now digital gadgets. How can I usefully sync actions across all of these objects.
- Physical things have digital representations: user reviews, histories, etc.
- Social machines: what does it mean when devices are participating in our digital lives? When our objects and places are participating in our digital lives. We don't want noisy interfaces in our lives: don't make everything full of personality.
- Software makes hardware scale. An endless variety of content can come from devices connected to people. The lifespan and durability of hardware can be extended through software.
- LG has created a chat interface for their appliances. You can chat with them to collect information and start/stop tasks.
- Connected devices won't always say nice things. They can be hacked like other devices.
- Cars, appliances, and more can have security and privacy issues we need to be mindful of as well.
- Software is ideology, embedded with values. The genie is out of the bottle, the systems are out there already. So it is up to us to do the right thing. Be conscious about the behavior your interfaces shape.
- Honor intention but don't assume it. Knowing the physical facts, isn't enough to assume what people want to do.
- We're already switching between devices. Plan for it in your designs.
- Don't just sync content and files, sync status, actions and processes.
- Peer to peer technologies (even in the browser) can make connections between devices.
- Think about how you can move interactions off of screens and onto sensors. It's not a challenge of technology.