kilowatts per terabyte, or terabytes per kilowatt: a measure of storage energy efficiency

A measure of the efficiency of data storage; the equivalent would be W/GB (watts per gigabyte).

The range measured below is 0.4 TB/KW (RAM) to 500 TB/KW (Blu-ray disks), with a corresponding range in access speed and update times.

Some estimates or measures:

If you are using solid state memory (no disk; just memory, configured as disk) from Texas Memory Systems, expect to have 2.5 KW/TB, or 0.4 TB/KW:

The Tera-RamSan incorporates specially designed 4-Gbit Fibre Channel or 4x InfiniBand interfaces that include performance-enhancing hardware to reduce I/O latency to under 14 microseconds. With I/O requests being served so quickly, an order-of-magnitude more requests per second are executed by a single Tera-RamSan interface as compared to other Fibre Channel interfaces. By scaling SSD modules up to 8 units (8 Fibre Channel links per unit or 4 InfiniBand links per unit), the Tera-RamSan is the ultimate storage solution for high-performance applications in total IOPS and total bandwidth. Also, because the Tera-RamSan is DDRRAM based, there is NO penalty for RANDOM accesses. Data requests need not be tuned for performance. User access time is not compromised, even if tens of thousands of users access the system simultaneously. Extreme performance is the key feature of the Tera-RamSan.

From 2005, Pierre Devries: including cooling about 1 KW/TB, or 1 TB/KW:

Working off Barroso, Dean and Hölzle’s paper on the Google cluster architecture, I infer that the energy consumption for storage is approximately 1W/GB: a dual 1.4-GHz Intel Pentium III server with a two 80GB drives draws 120W per server. Adding Google’s estimate of about 40% for cooling, gives 165 W per 160 GB, or 1 W/GB (watt/gigabyte).

in the comments to that post, using the archive.org architecture: 50W/TB, or 20 TB/KW:

Capricorn tech clains that their Petabox storage, which they built for archive.org, runs as little as 50W/TB. I don't think that number includes the power consumption of the required air conditioning.

updating Capricorn's Petabox to the current (2008) web site, we see 27W/TB, or 37 TB/KW:

The PetaBox is completely scalable, from individual terabyte nodes to petabyte clusters. A single 19-inch rack can support up to 120TB of raw disk space. This density is achieved through an economy of design that consumes as little as 27 watts per terabyte.

Computer Weekly in 2007 talks about metrics but doesn't actually measure:

An individual drive uses 5W to 15W of power depending on its capacity, rotation speed, form factor and operating state, but "you can't just multiply the number of drives in an array by some average power rating to get a total," says Mark Greenlaw, senior director with storage marketing at EMC. The power consumption of the array is more than the sum of the power used by the individual drives. Controllers and other components consume power. Copan Systems proposes two metrics for archival data storage: storage density measured in terabytes per square foot and terabytes per kilowatt.

A COPAN Systems press release from 2007; naturally, take press releases for what they are worth. The range from 4 TB/KW (Fibre Channel), 17 TB/KW (SATA), and 100 TB/KW (COPAN's sooper stuffs):

COPAN Systems has taken a "revolutionary" approach in how it has engineered its products over the past 4 years to deal with power and cooling issues in the data center. The low power requirements of COPAN's enhanced MAID architecture integrated with the extremely large storage density delivers in excess of an impressive 100 terabyte (TB) per kilowatt, the highest in the industry. This compares to an average power density for Fibre Channel disk products of approximately 4 TB per kilowatt and an average rate for standard SATA disk products of 17 TB per kilowatt. The savings are significant with estimates for power and cooling savings between $5.7 million and $7.8 million over a four year time period.

MAID, or "massive array of idle disks", if you don't know it, is an architecture where the drives are usually powered down; you wouldn't use it for active hot storage, but for mostly offline stuff it's a possible solution.

For the weird, here's a read-mostly solution: a Powerfile array of Blu-ray optical drives, at 500 TB/KW.

Appliance Leverages Next-Generation Blu-ray Disc™ Technology to Deliver 70 Terabytes per Rack, 120 Terabytes per Appliance, and Industry-Leading Energy Efficiency of 500 Terabytes per Kilowatt

Comments

[Edward]

updating some current numbers for the SSD disks (dec 2008):

from Texas Memory Systems:

RamSan-440: 1.28 KW/TB, 0.78 TB/KW
RamSan-500: 0.15 KW/TB, 6.7 TB/KW
RamSan-400: 2.8 KW/TB, 0.36 TB/KW

http://www.superssd.com

[Edward]

http://www.xyratex.com/Products/storage-systems/storage-F5404E.aspx

Xyratex F5405E fibre channel SATA RAID: 48 TB in 800W (Typical), or 60 TB/KW or 0.016 KW/TB.

Their literature makes a lot about their advanced power management abilities, e.g. spinning down idle drives, so I don't know what "typical" means relative to a real benchmark.

[Edward]

a happy COPAN MAID customer:

http://searchstorage.techtarget.com/news/article/0,289142,sid5_gci1338937,00.html

"The most obvious advantage for Copan is power and cooling savings from MAID. "I can fully load that thing, and I have 25 kW to 30 kW in my server and have almost 800 TB," he said. "We have 110 [SATA] disks in the tray, and in a year we've had one disk failure."

800 TB / 30 KW = 26.7 TB/KW = 0.0375 KW/TB

[Edward Vielmetti]

RamSan-620 highlights:

* 2-5 TB SLC Flash storage
* 250,000 IOPS random sustained throughput
* 3 GB/s random sustained throughput
* 325 watts power consumption
* Lower cost

or 5 TB/ 325 W which is about 15 TB/KW

http://www.superssd.com/products/ramsan-620.htm

[Edward Vielmetti]

COPAN 300M

896 Terabytes, max power consumption 6.2 KW

or 144 TB/KW

note that the max aggregate rated throughput is 5.2 TB/hour, so this is not intended for high performance main disks; it's for massive, mostly fixed, archival storage.

http://www.copansystems.com/products/native_maid.php

[Joe Landman]

Hi Edward:

Just a note, not a commercial.

Our 96TB 5U JackRabbit (http://scalableinformatics.com/jackrabbit) units consume roughly 1.1kW operating, and require ~0.33 ton AC to cool. I am not sure how efficient the AC is for cooling, so if we simply assume it takes 1.65kW to cool 1.1kw (assuming a 50% efficient AC, most are somewhat better than that), then actual storage vs power consumption is 96TB/2.75kW or 34TB/kW. Certainly the 87.3TB/kW number (ignoring cooling) sounds better, but it doesn't quite reflect actual costs.

I should point out that our units are quite fast, with a 24 drive (24TB) unit sustaining north of 1.5 GB/s writes, and more than that on reads, for large data sets, much larger than ram. Of course you can cluster them.

We are in Canton, roughly 20 minutes east of Ann Arbor.