Amazon Makes 1TB of Public Data Available for Developers

By Greg McNevin

February 27, 2009: In a move that could revolutionise internet data mashups as much as it revolutionised online retail, Amazon has given developers using its Elastic Compute Cloud (EC2) system easy, on demand access to over 1TB of data from public data sets.

Data from the US Bureau of Transportation Statistics, the English version of Wikipedia, the entire Freebase Data Dump and the DBPedia knowledge base has now added to EC2 alongside an already large number of significant genetic and scientific databases.

All the data is in machine-readable format, making it easy for developers to extrapolate and use in web applications, and do so in minutes as after a few short steps to setup the data sets, they can be pulled into applications instantaneously.

Instead of spending days or weeks downloading these data sets you can be up and running from a standing start in minutes,” writes Amazon CEO Jeff Bezos on the company’s Web Services Blog.

“Once again, cloud computing reduces the friction between "I have a good idea" and "here's the realization of my idea." You don't need loads of bandwidth, processing power, or local disk space in order to do interesting and significant work with these world-scale data sets.”

While quite US-centric, the breadth of data available at developer’s fingertips is likely to foster some incredibly unique, useful and innovative applications.

With all of the information on the human genome, all publically available DNA sequences, data on 2.6 million people not to mention places, films, albums and 20,000 companies, the possibilities for developers are simply tremendous.

Comment on this story