An Event Apart: The Immobile Web

by August 5, 2013

In his presentation at An Event Apart in Washington DC 2013, Jason Grigsby walked through the latest information about the Web on TV sets and the implications for Web designers and developers. Here's my notes from his talk on The Immobile Web.

  • Google TV features applications built with Web technologies. The experience feels like TV but behind the scenes these are just Web pages.
  • The Vimeo experience on Google TV is Web-based and well done. You can actually access it on the Web by clicking on the "couch mode". It's surprising how good this experience is.
  • The next wave of the zombie device apocalypse is going to be TVs. 88/86% of people with smartphones and tablets use them while TVs are on. Humans are the API between the TV and the devices in our hands. The dumb screen is on the wall and the smart screen is in your hand. If TVs were smarter they could do a lot of the work we're doing ourselves for us.
  • Netflix is 35% of downstream traffic on the Internet. Its so important that TVs ship with Netflix buttons on the remote.
  • XBox Live (online channel on the game console) is used more for entertainment (watching tV and movies) than gaming.
  • TV manufacturers are experimenting with all kinds of smarter TVs to fend off Apple's entry into the TV space. They are all afraid of being disrupted.
  • When Apple does get into the market, it is likely people will scramble to create TV experiences. The big opportunity on TVs are app stores. It will open up development on TVS to everyone like app stores did for mobile.
  • If we have apps on Apple TVs, then we'll have embedded Web views, and if we have embedded Web views, then someone will make a Web browser. If someone does, then Apple will build one too.
  • Considering TVs helps us make sure we are not being short sighted about our Web solutions. It helps us build for today and be ready for tomorrow.
  • The browsers on 2013 TVs are surprisingly good. They score well on HTML5 tests. 100% of smart TVs supported common CSS3 properties and a decent amount of more advanced features.
  • The 2012 and 2013 models of smart TVs are much faster due to better (dual, quad core) processors. This means rendering time is much better.
  • Xbox Smartglass is kind of like Airplay from Apple but a bit more advanced. It allows you to sync content on smartphones as videos are playing. The platform allows you to use Web technologies to create Smart Glass applications.

TV Input

  • The experience of using smart TVs is really bad -mostly because of input. The ability to browse the web and go to URLs is broken due to poor input capabilities.
  • Smartphone apps that allow you to control TV interfaces actually work much better for browsing and text input. Nearly every TV manufacturer has released these kinds of apps now.
  • It's a multi-screen world. We need to think about how these devices work together and what role will the Web play in this world.
  • Thinking about making your content accessible is only the first part of multi-device design.
  • How do you link a Web page on the TV to a Web page on your phone? We need to figure this out or else the Web will be behind apps again.
  • The ten foot experience requires a different approach to user interface. Make text easy to read on TVs: make font sizes bigger, make line heights bigger. A good rule of thumb is to make elements (like images or fonts) 1.5x bigger for 720p and 2.0x for 1080p relative to the size of that element in a normal PC browser.

Detecting TVs

  • Opera supports a spatial navigation feature that you can specify using CSS to help people move around a Web TV interface.
  • Google TV recommends using JQuery TV for building TV interfaces but this doesn't work everywhere. The Chrome team is now taking over development of the TV browser.
  • TV browsers support arrows keys fairly well. But remote controls are not detectable.
  • The low end iMac's screen resolution is 1080p just like a TV. But the interface for a TV should be very different from the interface for a desktop computer.
  • Actual resolutions on TVs may vary. Even when TV browsers support the full screen API, they may leave browser chrome on screen. We can't use resolution as an indicator of what's a TV.
  • Can we use CSS media types? No TVs on the market use the TV media type. Media types are all mutually exclusive: if you render one, you can't render the others. All Web sites use the screen media type. So all TV vendors render screen media types.
  • Issue like these stem from the actions of Web developers. We made everything work for screen and no that TV media types would be useful, they can't be counted on.
  • Can we use Javascript or user agent strings to detect TVs? The most reliable method right now is user agent strings so if you have to built Web pages for TVs now, you can use that. There's still gaps however but not too many.
  • We need to put pressure on TV manufacturers to standardize on Web development capabilities.
  • If you thought testing phones were bad, you haven't seen anything yet. Most of the TVs in stores aren't connected to WiFi, don't have remotes, and aren't up to date. Testing on these devices is more challenging.

Adaptive Input

  • Playstation 4 and Xbox One both feature new, powerful browsers.
  • There are three operating systems running on the Xbox in order to enable switching between TV, the Web, and games. This makes switching to the Web very quick and easy.
  • We've been too focused on screen resolutions and widths as an indicator of experience. These are not enough to tell us when to deliver a design appropriate for TVs.
  • We've had a consensual hallucination of the Web: that it was a fixed canvas size of 640x480 then 1024x768. In the past few years we've made a lot progress of not thinking like this. We've recognized the Web does not have a fixed width.
  • Now, though, we have another hallucination. That desktops are equal to mouse and keyboard and smartphones are just touch. Once again this isn't true. Smartphones and tablets have keyboards, cursors, and styluses. Touch screen laptop sales jumped 52% the last quarter.
  • We need to think about what it means to create multi-device experiences. How do these devices connect and interact?
  • The big challenge is adaptive input. Just like we don't have a fixed canvas for layout designs, we don't have a simple distinction of keyboard and mouse on big screens and touch on small ones.
  • In our industry we tend to look at desktop design and mobile design as requiring different interfaces.
  • Chrome thinks detecting touch and adapting your interface is a bug. They are contacting sites asking to stop using this technique.
  • Input is dynamic, people can change input types within a single flow or process.
  • You can respond to different input types when they are being used like making buttons bigger when someone touches the screen or smaller when someone moves a mouse. But we probably need to be more zen about input and embrace the fact it comes in different forms at different times.
  • Touch screens and mice or trackpads can be used together to create better interfaces than just one input at a time.
  • Keyboard and mouse targets require different sizes to be used comfortably. Touch targets need to be bigger. If you make an interface that is usable for touch, it will be usable with mice.
  • What about those who won't give up on the power users? Those who feel they're more productive with lots of things on the screen. Larger touch targets mean less things on the screen. So power user scenarios and complex user interfaces suffer, right?
  • First, touch will be on people's work machines in the immediate future. Five years from now will you not be able to buy a Windows computer without a touch screen.
  • Second, the real issue here is display density and not touch vs. mouse. Some people want to see more information on a display. That can be accomplished with a display setting. The same kind of display setting can be used to switch between lean back (10 foot) and lean in (18-24 inch) interface designs.
  • For example, GMail allows you to pick between comfortable and compact views of your desktop.
  • This kind of approach allows us to design for people's needs not for a specific form factor or display. This is device form factor and input agnostic design.
  • Adapting to input is not responsive design it might be more like progressive input: how do we adapt to the input people have on their device? But we don't need to have a value judgement: no one input type is necessarily better than others.
  • Instead we need to learn to adapt to what input is there. This is a new area, let's work on it together.

Lessons From TVs

  • Smart TVs suck right now. Don't go build something for them yet, but don't sell their future short based on current experiences.
  • We can't predict future behavior from a current experience that sucks.
  • Even if we don't have to design for TVs today, we need to be aware of where things are going.
  • TVs help prepare us for the uncertainty that is to come.