Wednesday, July 14, 2010

dd performance: block size matters

I was happily planning on using dd to wipe a drive empty, until this depressing statistic stared at me.

85231 byte/sec

A few (ok, many) furious secs of mental maths leaves me depressed at my having to wait 200+ hours to wipe a 72gb enterprise SCSI-3 drive. This can't be right!

Luckily, a light bulb at the back of my head reminded me that the dd command did have options that allowed specifying the block size: the bs=512 for example.

Anyway, I experimented with various block sizes from the presumably default 512 (bytes) all the way to 64mb, found the optimum value somewhere within the range, and stuck to it. The resultant best speed was approximately 9mb/sec.

Here I'm intentionally trying to leave out the optimum block size I discovered, as it is very likely to be system and disk dependent.

But what is clear, and in hindsight, pretty much common sense, is that too small a value, and the disk wastes too much time seeking to read/write small blocks. Too large a value, and other system inefficiencies come into play, such as buffers/caching/paging.

In short, if you're performing a long operation, it definitely pays to experiment first to find a more optimal value than the default.

No comments: