« Machines of Loving Grace, Richard Brautigan | Main | How to produce a map from a spreadsheet »

October 21, 2007

How to download all of your twitters into Excel

I'm scheming to make Twitter more productively personally useful, and as a part of that I'm finding a need to pull my archive of past twitters out into a format that's easy to parse with the tools that I have at my disposal.

A goal, then, is to write a command that generates a file which can be imported into a spreadsheet so that all of my spreadsheet-fu can be applied to it. (There isn't much spreadsheet-fu, but what there is is very handy; I'm particularly fond of interactively sorting through tables, and I long to be proficient in pivot tables.) The easiest way to write such a thing is to describe it enough detail that someone might have written it already, and then you can just use what they did.

The Google search terms would be
twitter (import OR export) (excel OR csv)
and I get 1.9 m documents, none of which are an exact hit, though there was a useful one about importing your Twitter contact list into Dopplr.

The del.icio.us tag looks like
http://del.icio.us/tag/twitter+export
but it doesn't get me what I want either, though I did discover that "social network" in German is Freundesnetzwerk.

Think a little bit, search for
Twitter API
and a few clicks later I'm at the Twitter Fan Wiki . That led me towards twitish, a Perl based shell for twitter. There's also python-twitter, a Python wrapper for the API.

that's fine but what I really just want is not to write code. Posting this as a draft, pls comment if you find it, otherwise I'll look again when next the urge hits.

Comments

Here is a PHP snippet of PHP that will do what you want I think... Unfortunately, twitter limits their returns to the most recent 20, so you'd need to continually run this to get them all... This is spitting it out in pipe delimited, cuz I was being lazy...


# Date: 10/22/2007
# Author: Mike Monan (monan@techreprieve.com)
# Description: Exports all twitters into pipe deliimited file...
# License: do whatever you want with it...

$csvfile = 'twitter.csv';
$csvf_res = fopen($csvfile, 'w') or die ("Can't open CSV file: $csvfile");

$feedUrl = 'http://twitter.com/statuses/user_timeline/monan.rss';
$feedUrl = 'http://twitter.com/statuses/user_timeline/monan.xml?count=200';
$rawFeed = file_get_contents($feedUrl);

# for debugging...
#fwrite($csvf_res, $rawFeed);

$xml = new SimpleXmlElement($rawFeed);
date_default_timezone_set("America/Detroit");

# Print a header...
fwrite($csvf_res, "RawDate|PrettyDate|TwitterText|MoreLink\n");

# Dump our data...
foreach ($xml->status as $rssitem)
{
$rssdate = strtotime($rssitem->created_at);
$rssprettydate = date("D, M j, Y h:i:s A", $rssdate);
$rssgoodstring = $rssitem->text;

# Escape any pipes, lazy way...
$rssgoodstring = str_replace('|', "", $rssgoodstring);

# We're going to limit using pipes, because I'm lazy and don't want to go through all the escape contortions...
fwrite($csvf_res, $rssdate . '|' . $rssprettydate . '|' . $rssgoodstring . '|' . (string)$rssitem->link . "\n");
}

fclose ($csvf_res)

Post a comment

If you have a TypeKey or TypePad account, please Sign In

My Photo

Subscribe to Vacuum

  • Subscribe with Bloglines

    See also my other blog, Superpatron, for library patrons and libraries.

Once the search has begun, something will be found

  • Google Custom Search

Vacuum archives

  • archives of vacuum - include things hosted on other sites. (not linked yet TBD checking style now) 1999: 1 2 3 4 5 6 7 8 9 10 11 12 2000: 1 2 3 4 5 6 7 8 9 10 11 12 2001: 1 2 3 4 5 6 7 8 9 10 11 12 2002: 1 2 3 4 5 6 7 8 9 10 11 12 2003: 1 2 3 4 5 6 7 8 9 10 11 12 2004: 1 2 3 4 5 6 7 8 9 10 11 12 2005: 1 2 3 4 5 6 7 8 9 10 11 12 2006: 1 2 3 4 5 6 7 8 9 10 11 12 2007: 1 2 3 4 5 6 7 8 9 10 11 12

Call me!

  • Call me!

upcoming.org

What I'm up to

mybloglog


103bees vacuum

Hit tail

Blog powered by TypePad
Member since 08/2003