In their presentation at the Google Chrome Dev Summit in Mountain View CA, Sam Dutton and Jan Linden walked through new media APIs available in modern Web browsers and what you can do with them. Here are my notes from their talk:
- Media APIs for the open Web work across a wide range of devices.
- WebRTC is an ambitious project to bring real-time communication to the Web. It's adopted in Chrome, FireFox, Opera, and more which adds up to a billion end points.
- Video chat is the most obvious use case for WebRTC but other innovative apps are being built now.
- WebRTC is peer to peer, which makes performance much better.
- MediaStream (GetUserMedia), RTCPeerConnection, RTCDataChannel are the Javascript APIs. RTCPeerConnection it manages signal processing, codec handling, bandwidth management, security, and more. RTCDataChannel uses the same API as WebSocket but with ultra-low latency. It can be optionally specified as reliable or unreliable. File sharing requires all bits are transported, live streaming might not.
- GetUserMedia is pretty straightforward. You ask for the type of media you want and, once successful, you can do what you want with the file you get back.
- You can use constraints to adjust resolution and source of media (like different cameras).
- Media Stream Capture proposes to give you access to auto-focus and zoom.
- Web Audio is well supported through the audio tag. But where is that not enough? When precise timing of multiple sounds, creation of audio and processing, and analysis of audio data. Web Audio allows all this and you can use the audio tag as input for Web Audio.
- Browser support for Web Audio is in Firefox, Chrome, Safari 6.0, and Opera.
- All these media APIs work together. For example, GetUserMedia can be a source for Web Audio.
- Web MIDI is a new standard that allows you to connect devices to media interfaces.
- Open and free video codecs for the Web are important. VP9 is an improvement on VP8 and uses significantly less bit rate.
- Media source extensions allow you to do adaptive streaming based on bandwidth changes.
- Videos can be encoded with alpha transparency and you can apply CSS transforms and transitions to them.