I have had a really hard time trying to find information regarding I/O
performance in .NET. All I wanted to know was the optimal buffer size for a
FileStream on an ordinary single disk machine, but I couldn't even
find that. As it happens, to day I stumbled across a
paper by Microsoft Research which talks about I/O performance with .NET v2
(beta 1) and answers a few important questions. It is not long, so I would
advise everybody to have a read through it; but for those that are really really
lazy here is the summary (quoted from the paper itself)
(1) For single disks, use the defaults of the .NET framework – they
deliver excellent performance for sequential file
access.
(2) Pre-allocate large sequential files (using the SetLength() method) when
the file is created. This typically
improves speed by about 13% when compared to a fragmented file.
(3) At least for now, disk arrays require un-buffered IO to achieve the
highest performance. Buffered IO can be 8x
slower than un-buffered IO. We expect this problem will be addressed in
later releases of the .NET framework.
(4) If you do your own buffering, use large request sizes (64KB is a good
place to start).
(5) Using the .NET framework, a single processor can read and write a disk
array at over 800 MBps using unbuffered
IO.
Another interesting fact (which contradicts the first point a bit as the
default is an 8K buffer, but makes sense if you read the whole paper) is
that a 64K buffer appears to be the answer to my question at the begining of
the post.
The paper also comes with code snippets which
can be dowloaded from here (search
the page for "Sequential File Programming in .Net" ... stupid
javascript)