Interesting Data Sets

These are some public data sets that might prove interesting for experimentation or exercise development.

• Wikimedia Data Dumps
• DBPedia
• ArXiv Articles - Bulk Data Access
• Million Song Dataset
• GeoNames Data Dump
• IP Geolocation Database
• NHTSA Office of Defects Investigation data
• Federal Election Commission Downloadable Data
• BioID Face Database
• 100,000+ Official Crossword Words
• 350,000+ Simple English Words
• Google Ngram Corpora
• Amex Exchange Daily Data: 1970-2010
• NASDAQ Exchange Daily Data: 1970-2010
• NYSE Exchange Daily Data: 1970-2010
• SEC EDGAR Filings Data
• IRS Tax data
• Amazon Product Review and other Sentiment Data
• US Census 2000 Tract Level Planning Database
• US Census TIGER Geographic Data
• US Census Frequently Occurring Names
• Movie Review Data from Cornell
• GroupLens / MovieLens Data Download
• Music Recommendation Datasets from Last.FM
• MusicBrainz Database Download
• MetaFilter Infodump
• Fair Market Rents Data from HUD
• Climate Data from Univ. of Delaware
• NOAA Daily Climate Data
• USGS Earthquake Data
• Supreme Court Database
• Enron e-mail Corpus
• Common Crawl data set (~ 6 billion documents from Web crawler)
• Foursquare data set (NOTE: the data set is not public, but this page describes it and provides contact info for requesting a copy)
• 100 million photos and videos from Yahoo/Flickr, under various permutations of the CC license.
• Reliability data for 41,000 hard drives