As part of watching DTE slowly restore power to the area, I've been pulling down copies of their outage maps. These are big PDF files, suitable for framing, that encode a bunch of data in them about where the power is out.
Here's some tools I used to get details out of PDF into image formats, (hopefully) to get them back down to a spreadsheet full of numbers.
1. Fetch the outage report automatically with curl:
DATESTAMP=$(date "+%Y-%m-%d-%H-%M")
curl -s -o outage.$DATESTAMP.pdf http://my.dteenergy.com/map/zipCodeOutageMap.pdf
This grabs the file and date/time stamps it appropriately. I'm running in an every 15 minutes loop, which means it grabs the file that's updated every 30 minutes twice (a little redundancy, not too bad).
2. Convert the report from PDF to PNG with sips:
sips -s format png outage.$DATESTAMP.pdf --out outage.$DATESTAMP.png
'sips' (the "scriptable image processing system") has a bunch of Applescript front end documentation for it, but there is also a command line lurking there as well. I'm collecting bookmarks for tutorials on sips at delicious. There's a lot that it can do, since it is mostly a front end for the core graphics library on the Mac which is quite powerful, but the doc is clearly not written with the intention that anyone uses it directly.
3. Do fun stuff to a series of images with ImageMagick:
composite -compose difference ~/Desktop/outage.2008-12-28-23-47.png outage.2008-12-29-13-28.png difference-progress.gif
ImageMagick has dozens, maybe hundreds, of transformations that you can use to take a set of images and turn them into something else. It's powerful and also reasonably well documented, and I won't even try to summarize the ImageMagick command line options list which gives you plenty to play with.
4. Stare at the results and try to make sense of them.
Here's the DTE progress map (middle) with before (left) and after (right) reflecting about 14 hours of restoration efforts. Click through to Flickr for detail. The large amounts of black in the middle photo represent locations where work progress has not resulted in a change in service that's visible, though there may in fact have been 1000s of lines restored.
5. What I want is a tool that would work on a PNG and would give me the color value of a given pixel, or that would give me a histogram of all of the color values of all of the pixels in an image. There is a histogram function in ImageMagick but it doesn't quite do the trick at first glance, and I'd be better off posting this than trying to do more myself!
The takeaway on this if you squint hard enough - 14 hours of effort, and not a lot of immediately visible progress across broad parts of Livingston, Washtenaw, and Downriver areas in DTE service restoration.
