My AOL data project has got­ten me both inter­ested in and ter­ri­bly frus­trated with the chal­lenges of work­ing with mas­sive amounts of data. Data manip­u­la­tion and opti­miza­tion is not my bag, but I’m afraid it’s going to have to be soon. In any case, I thought I’d share some of the great resources I’ve found.

Interesting Datasets

I would love to do projects with each of these.

• AOL Search Data
• Enron Company Emails
• The Entire Contents of Wikipedia
• Hillary Clinton’s Schedules as First Lady
• Public Transit Data Feeds
• World Database of Happiness

Compilations of Publicly Accessible Data

These sites all point to datasets that you can down­load and incor­po­rate into your projects.

• Pages tagged with “pub­lic­data” on del.icio.us
  Subscribe to the feed on this page to keep an eye on what inter­est­ing stuff oth­ers find.

• A Meta-index of Data Sets
  A great start­ing point, with more help­ful links than this post.

• Digitized Historical Collections from the Library of Congress
  The cen­tral start­ing point for all records at the Library acces­si­ble via the Open Archives Initiative Protocol. I’m not 100% sure what that means, but it sounds like some­thing I could use down the line.

• Infochimps
  A newly launched site with lots of poten­tial to grow into some­thing interesting.

• Copyright Free and Public Domain Media Sources
  Points mostly to pho­tographs and other images, and mostly older ones, at that. Thanks, Copyright Term Extension Act of 1998.