Webstock 2008: Designing for a Web of Data

by February 16, 2008

Tom Coates’ Designing for a Web of Data presentation at Webstock 2008 outlined a set of principles for approaching the design of applications that truly leverage the power of the Web’s vast data sets and integration capabilities.

A Web of Data

  • Geo-location services, social networks, and Web utilities have pushed the envelope: they are more than just Web sites. They are interfaces for set of data behind the scenes. Manifested elsewhere across Web. They are platforms that can be built on.
  • We are moving from a Web of pages to a Web of data. Pages are the human-facing aspects of the data and services behind the scenes. Data is being distributed anywhere the network can touch.
  • Web of Data: sources for exploring and manipulating data and ways people can plug them together.
  • Data is anything explained in a way that a computer can make sense of them. Your site is not your product
  • Your territory is anywhere your network touches.
  • Twitter can update online, SMS, desktop, IM. Anything that can broadcast information to twitter CAN send information to Twitter. 90% of Twitter’s traffic comes from APIs.
  • Flickr can deliver photos to digital photo frames, widgets, badges, phone updaters/downloader’s, desktop clients, mash-ups.
  • Last.fm is a Web site & client download that plays audio.
  • As network touches more things, the store of data is accessible everywhere. Physical objects can visualize data form the network. Interacting with physical objects allows people to directly change data stored in the network. Physical objects can act as sensors that write to a Web of data.

You must play well together

  • It’s good to design for recombination.
  • Open up data and services to drive people to you. People will pay for services (especially with good data). Put yourself in the middle of an ecosystem.
  • Network effects: every new service can build on top every other existing service. Every new service or piece of data you add makes all services better.
  • Combining data together makes both more valuable.
  • Fire Eagle: a store for location information. Let people know where you are. Has APIs for every application to add your location or read your location. Every Web application can respond to where you are and change as a result of it. Makes other services better.

You can never have too much data

  • Open up a dataset you own, build a dataset with users (perhaps the best way), or enhance one dataset with others.
  • Scale is the biggest problem you reach with a data service.
  • 2 billion photos in Flickr. One photo for every three people in the world. 100,000 photos added to Flickr per hour.
  • Way to allow people to make their way through huge sets of data is with more data. Capture data from production of objects, gather data from analyzing objects, crowd-source data, and derive from behavior.
  • Flickr: manifest and allow people to explore in many ways. From user, by time, curated by creator, curated by editors, geography, camera, licensed information, accessed from camera meta-data, and more.
  • Make data explore-able across all dimensions.

Hierarchies can’t take the weight

  • Move caused by the scale of data. Shift from hierarchic to Web-like exploration.
  • Amazon tabs: give up on hierarchies in top-level navigation. Instead explorations occur within pages using tags, collaborative filtering, lists built by people.
  • Top navigation is just a jumping of point