An Event Apart: HTML5 APIs

by Luke Wroblewski November 11, 2012

At An Event Apart in San Francisco CA 2012, Jen Simmons walked through new capabilities coming to the Web and their impact in her presentation HTML5 APIs Will Change the Web: And Your Designs. Here's my notes from her talk:

  • HTML5 coverage tends to focus on new elements: articles, section, input types, etc. However, there’s a whole additional set of technologies coming that Web creators need to be aware of: HTML5 APIs and beyond.
  • Infrastructure can stick around for a long time because its human nature to get stuck on the things we know. However, the Web is continually evolving and we need to evolve with it.

New APIs

  • HTTP allows computers to connect to servers, which send files (HTML, CSS, images, etc.) down to clients. It’s a back and forth process. HTTP doesn’t natively support continuous connections. Once something is sent down, the client is no longer connected to the server so data and interactions can’t flow seamlessly.
  • Web Sockets can augment HTTP to create truly bi-directional connections that keep data flowing between a client and server. It also works with multiple clients (and/or devices) so you can enable real-time cross device interactions. HTTP is like using walkie-talkies for communication, Web sockets are like using phones.
  • Servers traditionally store data for Web sites and applications. In order to read or write this data, clients need to connect to servers.
  • Today its possible to set up a database on the client, not just on the server. Data can be saved to and read from these client-side databases. Local storage, session storage, and Index DB are early examples of local databases.
  • One of the big assumptions of the Web is that you have to be online to use it. In the future, though, more interactions will be possible offline. Cache manifest allows you to store assets and information on the client for offline access.
  • The files on your local system don’t integrate easily with Web browsers. But now there are seven new APIs for dealing with file systems on the Web: file API, reader/writer/system, blob URLs & builder, drag and drop. Through these APIs files cannot only be used in the browser but they can be parsed, manipulated, written, and saved as well. Even information inside files can be extracted from in the client: metadata, file sizes.
  • getUserMedia is a Javascript API that collects and creates audio and video media streams from peripherals (camera, microphone) in the Web browser. With media streams, you can display and manipulate real time feeds of video or audio in web pages.
  • Web audio allows you to play files in the browser but you can also make use of effects like reverb, echo, etc. through primitives. People can create and share these primitives. This allows your primitives to be applied to other people’s audio.
  • Device APIs: allow you to connect to device capabilities like vibration, battery status, contact lists, and more in the Web browser.
  • Canvas: a space on a Web page where you can draw shapes, colors, etc. It can also work invisibly (not on a locked screen window) to do calculations behind the scenes for drawing elements.
  • WebGL: an API to do 3D drawing in the Web browser.
  • Controllers: not everyone has a keyboard and mouse. Web browsers are now available in video game consoles that have unique input controllers, as well as touch screens and more. Controllers allow you to use the full range of input on these devices.
  • Web MIDI API: allows you to interact with keyboard and MIDI input/output.

New Forms of Content

  • We’re creating content in RGB for devices. You can make more colors in RGB than CMYK. You can’t print certain colors in CMYK. What a photograph can be on the screen is very different from what it can be in print. What does it mean to shoot photographs for screens rather than for print?
  • Animated data visualizations are a form of communication that isn’t possible in print but is possible on screens.
  • Is the medium the message? The technology is not the content but the content is influenced by how it is shared with people. Don’t get caught up in the technology, but know how it can be used to effect and enhance your message.
  • There are too many Web designers not designing for the Web.
  • The Internet is just getting started. An innovator is someone who wakes up to constraints caused by false assumptions and breaks out of them.
  • More changes are coming, new APIs and beyond. What will you do with this new power?