Harddisks: Seek, Read, Write, Read, Write, Slow ?

Hi,

Take these 3 concepts and then look at the implementation/performance:

( 3 Concepts for reading/writing with harddisks )

Concept 1:

Seek, Read, Read, Read, Read, Read, Etc

Results: FAST

Concept 2:

Seek, Write, Write, Write, Write, Write, Etc

Results: FAST

Concept 3:

Seek, Read, Write, Read, Write, Read, Write,

Results: SLOW ???

The pseudo code is like:

Seek( 0 ); // offset 0

For I:=0 to FileBlocks-1 do // number of 4KB blocks in file.
begin
if Random(2) = 0 then
Read( 4 KB )
Else
Write( 4 KB );
end;

The original concept is:

Do a seek once.

Then read or write a block of data. The head is automatically forwarded to
the next block. So no extra seek is needed.

For concept 1 and concept 2 this works just fine and gives good performance.

However concept 3 has very bad performance.

Is this a software issue ? ( Windows XP )

Is this a hardware issue ? ( Harddisk Read Head and Harddisk Write Head
can't work together like this and an extra seek is needed ? )

Or some sort of driver issue ? ( Harddisk driver / firmware issue ? )

Bye,
Skybuck.

In windows xp I always have harddisk write cache off to prevent data loss in
case of power failure etc.

But just for the kick of it I enabled it, and I did no reboot. It had no
effect on performance, though I only tested shortly

Tomorrow I might test it once more... but after enabling it and before
testing it I will do a reboot.

I doubt I will see any performance difference.

If I don't see any performance difference in writing speed etc... that would
be a little bit suprising...

Since that would mean write cache doesn't help and can only cause loss of
data anyway.

Besides from that the document doesn't make any differences in Read and
Write speed.

My test programs clearly show there is a difference in Read and Write speed.

The document just mentions 'Data Transfer' speed... which is probably only
the Read speed.

The funny thing is my programs achieve those speeds roughly... so that's
correct... and my programs are correct :P

Hmmm...

I examined the source code of my read and write benchmark programs

The read program reads the totaly file.

So it does

1 Seek

X reads until file is read.

The write program however uses a cache of 1 MB.

So it does

1 Seek

X writes until cache is written and then again

1 Seek

X writes until cache is written etc etc.

So that can explain why writing is slower.

So I will have to write some new benchmarks... which behave exactly the same
for reading and writing

Bye,
Skybuck.

Skybuck Flying wrote:
Hmmm...
snip
1 Seek
snip

1 seek that you instructed it to complete. This is a multitasking operating
system.

Cheers,
Nicholas Sherlock

"Nathan McNulty" wrote in message
Actually, what is happening is that the heads on the platter need to
reconfigure themselves each time you read or write.

It is a hardware limitation.

By reading, the head can stay in the same location, but once it is told to
write, the head has to position itself to change the magnetism of the
sector of the platter it is on.

And that's why there is a write-to-read recovery (w-r) field in the make
up of every sector to allow the R/W channel to switch and the servo
to adjust the write-to-read element offset to center the write head .
http://www.hgst.com/hdd/ipl/oem/tech/noid.htm

Then once you request to read again, the head has to move again.

That's actually automatic as the drive reads every servo field and
every sector to keep track of where it's at.

It is the same as seek time

Nonsense.

when you have two files on different partitions it is going to take
longer to find those files than if they were right next to each other.

Nope. And read-ahead cache will have the contents anyway.


Nathan McNulty

kony wrote:

On Wed, 14 Jul 2004 17:53:52 +0200, "Skybuck Flying" wrote:


Hi,

Take these 3 concepts and then look at the implementation/performance:

[snip]

"Nathan McNulty" wrote in message
...
Actually, what is happening is that the heads on the platter need to
reconfigure themselves each time you read or write. It is a hardware
limitation. By reading, the head can stay in the same location, but
once it is told to write, the head has to position itself to change the
magnetism of the sector of the platter it is on. Then once you request
to read again, the head has to move again. It is the same as seek time
when you have two files on different partitions it is going to take
longer to find those files than if they were right next to each other.

Yes, a hardware limitation seems to make more sense.

I changed the code to this pseudo code, to illiminate the randomness and
make it regular switching:

if BooleanRead then
begin
Read 4 KB
BooleanRead := false;
end else
begin
Write 4 KB
BooleanRead := true;
end;

This will cause regular switching like:

Seek, Read, Write, Read, Write, Read, Write, etc

Instead of randomness like:

Seek, Read, Read, Write, Read, Write, Write, etc

My harddisk performs as follows for these 3 concepts (with the updated
concept 3 )

Concept 1 ( Reading )

It achieves 8 MByte/Sec to 12 MByte/Sec for reading a 36 MB file. ( with 4
KB block buffer )

After a few seconds the harddisk led stops burning. The speed increases to
100 or 180 MByte/Sec.

This is ofcourse only possible because it's fetched from some RAM I think. I
am not sure if it is coming from the Harddisk Read Cache or Windows XP file
cache... or whatever it's called. My guess would be Windows XP's file cache.

Concept 2 (Writing )

This test is very fluctating... sometimes 600 Kbyte/sec at worst...
sometimes a peak to 36 MByte/Sec.

Mostly it's 2 MByte/Sec with some peaks to 4 MByte/Sec.

My file system is pretty fragmented though.

Concept 3 ( Reading / Writing )

Well this is also pretty weird.

Sometimes it's 80 KByte/sec to 400 KByte/Sec.

Now it's 1 to 2 MByte/Sec

I suspect the small file size of only 36 MByte has to do with that.

So I will now test again on 100 MB file, to prevent any caching or to detect
it... possibly memory jump in task manager.

Concept 1 (Reading)

This time the speed is pretty constant. 8 MByte/Sec... with some lower peaks
to 6MByte/Sec and a few high peaks to 9 MByte/Sec.

But no more caching apperently in windows xp...

8 MByte/Sec or any multiple of MByte/Sec is what I would call good
performance.

This is in the light of my file transfer tool and 100 Megabit network cards
which are common nowadays.

100 Megabit is roughly 8 MByte/Sec or so... give or take a few...

Not that my file transfer tool reaches these speeds just... it has problems
with harddisk =D

One more artificact of my benchmark programs after stop the test it takes a
while stop... probably because of the for loop... no big deal.

Concept 2 ( Writing )

Well this time writing was a lot thougher... 1 MByte/Sec to max 2
MByte/Sec... the program is less responsive than the read program... but
code is almost the same... and multi threaded. But not using async stuff...
just sync stuff At least I think so However async stuff aint that
great either
Since I wrote a program to do async stuff as well. So never mind that :P

Now comes the big test, since it's getting late :P

Concept 3 (Reading and Writing, with regular switches )

Oh my... now that's what I call bad performance !

The first 10 seconds it was 80 Kbyte/Sec... the next 20 seconds 160
Kbyte/sec.

Max was 200 KByte/sec.

So it seems switching from read to write to read to write requires extra
seeking etc...

Maximum seek time for my HD and most HD's is 20 milliseconds.

Average Seek time is around 10 to 15 milliseconds.

So that means 50 seeks per sec for worst case.

The buffer is 4 KB...

50x4 KB = 200 KB.

That's pretty amazing... the harddisk performs even worse than the worst
case scenerio.

I did not expect that.

Well I found this document about my harddisk...

It contains some information about seek times and such.

However the term 'Head Switch' means something else in this case.

It means moving from track to next track or something I think...

As far as I could tell there is no mentioning of latency from read to write
to read to write etc...

Or maybe I missed it... Anyway time for me to call it a day.

http://www.hitachigst.com/tech/techl...56A7A006F9551/$file/dtta_sp.pdf

Bye,
Skybuck.




















































This clearly indicates the file



With the new code my harddisk now performs in the range of:

80 KByte / Sec to 400 KByte / Sec which I call poor/bad performance.

Concept 1 ( The Reading ) can achieve easily 8 MByte/Sec to 12 MByte/Sec.


After a few seconds on a 36 MB file it achieves 180 MByte/Sec this is
probably from Windows XP cache or maybe Harddisk Cache ?

The harddisk is completely silent. I can't imagina th






can achieve 1 MByte / Sec up to 12 MByte/Sec














Nathan McNulty

kony wrote:

On Wed, 14 Jul 2004 17:53:52 +0200, "Skybuck Flying"
wrote:


Hi,

Take these 3 concepts and then look at the implementation/performance:

( 3 Concepts for reading/writing with harddisks )

Concept 1:

Seek, Read, Read, Read, Read, Read, Etc

Results: FAST

Concept 2:

Seek, Write, Write, Write, Write, Write, Etc

Results: FAST

Concept 3:

Seek, Read, Write, Read, Write, Read, Write,

Results: SLOW ???

The pseudo code is like:

Seek( 0 ); // offset 0

For I:=0 to FileBlocks-1 do // number of 4KB blocks in file.
begin
if Random(2) = 0 then
Read( 4 KB )
Else
Write( 4 KB );
end;

The original concept is:

Do a seek once.

Then read or write a block of data. The head is automatically forwarded
to
the next block. So no extra seek is needed.

For concept 1 and concept 2 this works just fine and gives good
performance.

However concept 3 has very bad performance.

Is this a software issue ? ( Windows XP )

Is this a hardware issue ? ( Harddisk Read Head and Harddisk Write Head
can't work together like this and an extra seek is needed ? )

Or some sort of driver issue ? ( Harddisk driver / firmware issue ? )

Bye,
Skybuck.




I may not know the answer but feel an important question might be
"What is very bad performance?", compared to good performance...
numbers are our friends.

Could it simply be that you're switching back and forth with data
flow so the caching (on the drive) isn't effective?

Actually, what is happening is that the heads on the platter need to
reconfigure themselves each time you read or write. It is a hardware
limitation. By reading, the head can stay in the same location, but
once it is told to write, the head has to position itself to change the
magnetism of the sector of the platter it is on. Then once you request
to read again, the head has to move again. It is the same as seek time
when you have two files on different partitions it is going to take
longer to find those files than if they were right next to each other.

Nathan McNulty

kony wrote:

On Wed, 14 Jul 2004 17:53:52 +0200, "Skybuck Flying"
wrote:


Hi,

Take these 3 concepts and then look at the implementation/performance:

( 3 Concepts for reading/writing with harddisks )

Concept 1:

Seek, Read, Read, Read, Read, Read, Etc

Results: FAST

Concept 2:

Seek, Write, Write, Write, Write, Write, Etc

Results: FAST

Concept 3:

Seek, Read, Write, Read, Write, Read, Write,

Results: SLOW ???

The pseudo code is like:

Seek( 0 ); // offset 0

For I:=0 to FileBlocks-1 do // number of 4KB blocks in file.
begin
if Random(2) = 0 then
Read( 4 KB )
Else
Write( 4 KB );
end;

The original concept is:

Do a seek once.

Then read or write a block of data. The head is automatically forwarded to
the next block. So no extra seek is needed.

For concept 1 and concept 2 this works just fine and gives good performance.

However concept 3 has very bad performance.

Is this a software issue ? ( Windows XP )

Is this a hardware issue ? ( Harddisk Read Head and Harddisk Write Head
can't work together like this and an extra seek is needed ? )

Or some sort of driver issue ? ( Harddisk driver / firmware issue ? )

Bye,
Skybuck.




I may not know the answer but feel an important question might be
"What is very bad performance?", compared to good performance...
numbers are our friends.

Could it simply be that you're switching back and forth with data
flow so the caching (on the drive) isn't effective?

As I suspected the 100 MB file I was using to test... is pretty heavily
fragmented

I found this nice little too to find out if it's fragmented.

http://www.sysinternals.com/ntw2k/info/defrag.shtml

This was the output

: n \test files\bufferlog.txt

Clusters for file: C:\test files\bufferlog.txt
VCN: 0 LCN: 1300746 LEN: 16
VCN: 16 LCN: 1346392 LEN: 32
VCN: 48 LCN: 1345381 LEN: 64
VCN: 112 LCN: 2295765 LEN: 128
VCN: 240 LCN: 2510809 LEN: 219
VCN: 459 LCN: 2619481 LEN: 85
VCN: 544 LCN: 2467607 LEN: 218
VCN: 762 LCN: 2326190 LEN: 102
VCN: 864 LCN: 1866455 LEN: 218
VCN: 1082 LCN: 2778070 LEN: 102
VCN: 1184 LCN: 241057 LEN: 225
VCN: 1409 LCN: 2642790 LEN: 95
VCN: 1504 LCN: 2765237 LEN: 217
VCN: 1721 LCN: 2053603 LEN: 119
VCN: 1840 LCN: 403797 LEN: 217
VCN: 2057 LCN: 1280178 LEN: 119
VCN: 2176 LCN: 1870824 LEN: 225
VCN: 2401 LCN: 2483382 LEN: 111
VCN: 2512 LCN: 1184381 LEN: 227
VCN: 2739 LCN: 1280035 LEN: 125
VCN: 2864 LCN: 2559019 LEN: 229
VCN: 3093 LCN: 2764708 LEN: 123
VCN: 3216 LCN: 2058491 LEN: 233
more ('q' to quit):
VCN: 3449 LCN: 1382934 LEN: 119
VCN: 3568 LCN: 404503 LEN: 243
VCN: 3811 LCN: 2522295 LEN: 125
VCN: 3936 LCN: 857266 LEN: 243
VCN: 4179 LCN: 1179615 LEN: 126
VCN: 4305 LCN: 2176930 LEN: 243
VCN: 4548 LCN: 1281750 LEN: 126
VCN: 4674 LCN: 1465119 LEN: 213
VCN: 4887 LCN: 1184609 LEN: 173
VCN: 5060 LCN: 1866681 LEN: 250
VCN: 5310 LCN: 700707 LEN: 130
VCN: 5440 LCN: 700422 LEN: 252
VCN: 5692 LCN: 1822219 LEN: 132
VCN: 5824 LCN: 1284880 LEN: 255
VCN: 6079 LCN: 2514075 LEN: 146
VCN: 6225 LCN: 2456167 LEN: 259
VCN: 6484 LCN: 2559526 LEN: 144
VCN: 6628 LCN: 2178713 LEN: 260
VCN: 6888 LCN: 2313913 LEN: 153
VCN: 7041 LCN: 240583 LEN: 263
VCN: 7304 LCN: 2501987 LEN: 153
VCN: 7457 LCN: 2204221 LEN: 271
VCN: 7728 LCN: 1871065 LEN: 147
VCN: 7875 LCN: 1158481 LEN: 292
more ('q' to quit):
VCN: 8167 LCN: 2391566 LEN: 126
VCN: 8293 LCN: 2284425 LEN: 304
VCN: 8597 LCN: 2179820 LEN: 110
VCN: 8707 LCN: 2560656 LEN: 322
VCN: 9029 LCN: 2812066 LEN: 93
VCN: 9122 LCN: 2166757 LEN: 328
VCN: 9450 LCN: 2177179 LEN: 88
VCN: 9538 LCN: 90173 LEN: 335
VCN: 9873 LCN: 543314 LEN: 80
VCN: 9953 LCN: 2559672 LEN: 338
VCN: 10291 LCN: 2791169 LEN: 78
VCN: 10369 LCN: 1871243 LEN: 342
VCN: 10711 LCN: 1792079 LEN: 74
VCN: 10785 LCN: 180051 LEN: 346
VCN: 11131 LCN: 1598333 LEN: 69
VCN: 11200 LCN: 2188595 LEN: 351
VCN: 11551 LCN: 2209458 LEN: 65
VCN: 11616 LCN: 2179948 LEN: 352
VCN: 11968 LCN: 2063382 LEN: 63
VCN: 12031 LCN: 2218857 LEN: 17
VCN: 12048 LCN: 2167213 LEN: 353
VCN: 12401 LCN: 2068155 LEN: 78
VCN: 12479 LCN: 2325918 LEN: 17
VCN: 12496 LCN: 2177854 LEN: 365
more ('q' to quit):
VCN: 12861 LCN: 2479980 LEN: 65
VCN: 12926 LCN: 2589648 LEN: 34
VCN: 12960 LCN: 2168480 LEN: 383
VCN: 13343 LCN: 2481639 LEN: 63
VCN: 13406 LCN: 2600003 LEN: 34
VCN: 13440 LCN: 2392009 LEN: 384
VCN: 13824 LCN: 2458964 LEN: 62
VCN: 13886 LCN: 2609991 LEN: 34
VCN: 13920 LCN: 2059631 LEN: 205
VCN: 14125 LCN: 1643586 LEN: 51
VCN: 14176 LCN: 2293193 LEN: 389
VCN: 14565 LCN: 2167968 LEN: 72
VCN: 14637 LCN: 2591060 LEN: 51
VCN: 14688 LCN: 2171942 LEN: 391
VCN: 15079 LCN: 2480746 LEN: 69
VCN: 15148 LCN: 2502200 LEN: 68
VCN: 15216 LCN: 2219938 LEN: 394
VCN: 15610 LCN: 2204502 LEN: 82
VCN: 15692 LCN: 2582807 LEN: 68
VCN: 15760 LCN: 1158805 LEN: 399
VCN: 16159 LCN: 2168041 LEN: 76
VCN: 16235 LCN: 2798382 LEN: 85
VCN: 16320 LCN: 2170575 LEN: 203
VCN: 16523 LCN: 2799079 LEN: 85
more ('q' to quit):
VCN: 16608 LCN: 1708210 LEN: 203
VCN: 16811 LCN: 2815815 LEN: 85
VCN: 16896 LCN: 157273 LEN: 404
VCN: 17300 LCN: 854129 LEN: 102
VCN: 17402 LCN: 2829296 LEN: 102
VCN: 17504 LCN: 2883531 LEN: 419
VCN: 17923 LCN: 2809129 LEN: 87
VCN: 18010 LCN: 2766550 LEN: 105
VCN: 18115 LCN: 2287803 LEN: 199
VCN: 18314 LCN: 2456794 LEN: 118
VCN: 18432 LCN: 2176267 LEN: 423
VCN: 18855 LCN: 2306638 LEN: 98
VCN: 18953 LCN: 1994350 LEN: 120
VCN: 19073 LCN: 1280453 LEN: 200
VCN: 19273 LCN: 2168119 LEN: 138
VCN: 19411 LCN: 2284221 LEN: 197
VCN: 19608 LCN: 1868690 LEN: 150
VCN: 19758 LCN: 2491310 LEN: 187
VCN: 19945 LCN: 2483073 LEN: 151
VCN: 20096 LCN: 2594851 LEN: 183
VCN: 20279 LCN: 1868527 LEN: 156
VCN: 20435 LCN: 2172345 LEN: 200
VCN: 20635 LCN: 2623014 LEN: 151
VCN: 20786 LCN: 2479269 LEN: 200
more ('q' to quit):
VCN: 20986 LCN: 2300299 LEN: 156
VCN: 21142 LCN: 2186850 LEN: 192
VCN: 21334 LCN: 2512740 LEN: 175
VCN: 21509 LCN: 1753435 LEN: 193
VCN: 21702 LCN: 1281897 LEN: 177
VCN: 21879 LCN: 2558254 LEN: 194
VCN: 22073 LCN: 1530465 LEN: 177
VCN: 22250 LCN: 1283329 LEN: 188
VCN: 22438 LCN: 2209694 LEN: 429
VCN: 22867 LCN: 242236 LEN: 164
VCN: 23031 LCN: 240867 LEN: 178
VCN: 23209 LCN: 2869945 LEN: 436
VCN: 23645 LCN: 2479534 LEN: 178
VCN: 23823 LCN: 2789327 LEN: 178
VCN: 24001 LCN: 1282077 LEN: 179
VCN: 24180 LCN: 2314613 LEN: 180
VCN: 24360 LCN: 2871359 LEN: 442
VCN: 24802 LCN: 2513850 LEN: 129
VCN: 24931 LCN: 1556977 LEN: 465
VCN: 25396 LCN: 2176692 LEN: 110
VCN: 25506 LCN: 89547 LEN: 467
VCN: 25973 LCN: 241313 LEN: 4
Enumerate file clusters: STATUS_SUCCESS

The BufferLog.TXT was extracted from a WinZip file...

So I see two possibilities:

1. WinZip causes lot's of fragmentation

or

2. The free space was already fragmented...

I think case 2 was the case

Since I first deleted a compressed file etc...

And I know I deleted many files before making the compressed file etc...

So in other words... the free space was already pretty much fragmented

It would be nice to have a simple little tool... just to defragment a single
file for testing purposes...

Also I wonder about the Logic Cluster Numbers...

How does that work... performance wise ?

Are sequential Logic Cluster Numbers always garanteed to achieve maximum
performance ?

That would be amazing... so I don't think that's necessarily the case...

I read something about harddisk liking to writing/filling as many clustors
in a single 'cylinder'... a hollow cylinder that is... only writing the
rings of the 'cylinder'

At least that's how I imagine it could work:

For example:

Platter 1. Inner track 0 is filled.

Platter 2. Inner track 0 is filled

Platter 3. Inner track 0 is filled.

/\
| this is what they call a cylinder i think...
\/

Then when all tracks 'vertically' are filled.

Platter 1. Inner track 1 is filled

Platter 2. Inner track 1 is filled

Platter 3. Inner track 1 is filled.

Etc.

This should be done to minimize track to track movement etc...

So I wonder how windows xp 'distributes' the logic cluster numbers across
the disk...

Does it first fill up all tracks on a platter before proceeding to the next
platter...

Like so:

Case 1:

Platter 1, Track 1, Track 2, Track 3, Track 4, Track 5

Or does it use the 'cylinder' performance trick

Case 2:

Platter 1, Track 1, Track 4

Platter 2, Track 2, Track 5

Platter 3, Track 3, Track 6

Etc

That would be something... that would mean Windows XP optimizes
cluster/track placement for each disk differently... since each disk has
different geometry properties...

Whatever the case is...

I will assume placing all logical cluster numbers sequantially will still
give much better performance than randomly.

Skybuck.

Hmm...

Here is a little link about harddisk's layout and performance optimizations
though it's kinda old.

http://home.no.net/tkos/info/hd.html

So what I am interested in...

Is a little bit of information about

Harddisk numbering in general...

Like sector numbering, track numbering, platter numbering, head numbering,
ring numbering, etc.

How it corresponds to the physical location...

And ofcourse how harddisk read these

Hmm.. further more it seems... this numbering is called

Logical Block Addressing and it's now the dominant form of addressing the
disk

http://www.pcguide.com/ref/hdd/bios/modesLBA-c.html

"Nathan McNulty" wrote in message
...
Actually, what is happening is that the heads on the platter need to
reconfigure themselves each time you read or write. It is a hardware
limitation. By reading, the head can stay in the same location, but
once it is told to write, the head has to position itself to change the
magnetism of the sector of the platter it is on. Then once you request
to read again, the head has to move again. It is the same as seek time
when you have two files on different partitions it is going to take
longer to find those files than if they were right next to each other.

Yes, a hardware limitation seems to make more sense.

I changed the code to this pseudo code, to illiminate the randomness and
make it regular switching:

if BooleanRead then
begin
Read 4 KB
BooleanRead := false;
end else
begin
Write 4 KB
BooleanRead := true;
end;

This will cause regular switching like:

Seek, Read, Write, Read, Write, Read, Write, etc

Instead of randomness like:

Seek, Read, Read, Write, Read, Write, Write, etc

My harddisk performs as follows for these 3 concepts (with the updated
concept 3 )

Concept 1 ( Reading )

It achieves 8 MByte/Sec to 12 MByte/Sec for reading a 36 MB file. ( with 4
KB block buffer )

After a few seconds the harddisk led stops burning. The speed increases to
100 or 180 MByte/Sec.

This is ofcourse only possible because it's fetched from some RAM I think. I
am not sure if it is coming from the Harddisk Read Cache or Windows XP file
cache... or whatever it's called. My guess would be Windows XP's file cache.

Concept 2 (Writing )

This test is very fluctating... sometimes 600 Kbyte/sec at worst...
sometimes a peak to 36 MByte/Sec.

Mostly it's 2 MByte/Sec with some peaks to 4 MByte/Sec.

My file system is pretty fragmented though.

Concept 3 ( Reading / Writing )

Well this is also pretty weird.

Sometimes it's 80 KByte/sec to 400 KByte/Sec.

Now it's 1 to 2 MByte/Sec

I suspect the small file size of only 36 MByte has to do with that.

So I will now test again on 100 MB file, to prevent any caching or to detect
it... possibly memory jump in task manager.

Concept 1 (Reading)

This time the speed is pretty constant. 8 MByte/Sec... with some lower peaks
to 6MByte/Sec and a few high peaks to 9 MByte/Sec.

But no more caching apperently in windows xp...

8 MByte/Sec or any multiple of MByte/Sec is what I would call good
performance.

This is in the light of my file transfer tool and 100 Megabit network cards
which are common nowadays.

100 Megabit is roughly 8 MByte/Sec or so... give or take a few...

Not that my file transfer tool reaches these speeds just... it has problems
with harddisk =D

One more artificact of my benchmark programs after stop the test it takes a
while stop... probably because of the for loop... no big deal.

Concept 2 ( Writing )

Well this time writing was a lot thougher... 1 MByte/Sec to max 2
MByte/Sec... the program is less responsive than the read program... but
code is almost the same... and multi threaded. But not using async stuff...
just sync stuff At least I think so However async stuff aint that
great either
Since I wrote a program to do async stuff as well. So never mind that :P

Now comes the big test, since it's getting late :P

Concept 3 (Reading and Writing, with regular switches )

Oh my... now that's what I call bad performance !

The first 10 seconds it was 80 Kbyte/Sec... the next 20 seconds 160
Kbyte/sec.

Max was 200 KByte/sec.

So it seems switching from read to write to read to write requires extra
seeking etc...

Maximum seek time for my HD and most HD's is 20 milliseconds.

Average Seek time is around 10 to 15 milliseconds.

So that means 50 seeks per sec for worst case.

The buffer is 4 KB...

50x4 KB = 200 KB.

That's pretty amazing... the harddisk performs even worse than the worst
case scenerio.

I did not expect that.

Well I found this document about my harddisk...

It contains some information about seek times and such.

However the term 'Head Switch' means something else in this case.

It means moving from track to next track or something I think...

As far as I could tell there is no mentioning of latency from read to write
to read to write etc...

Or maybe I missed it... Anyway time for me to call it a day.

http://www.hitachigst.com/tech/techl...56A7A006F9551/$file/dtta_sp.pdf

Bye,
Skybuck.




















































This clearly indicates the file



With the new code my harddisk now performs in the range of:

80 KByte / Sec to 400 KByte / Sec which I call poor/bad performance.

Concept 1 ( The Reading ) can achieve easily 8 MByte/Sec to 12 MByte/Sec.


After a few seconds on a 36 MB file it achieves 180 MByte/Sec this is
probably from Windows XP cache or maybe Harddisk Cache ?

The harddisk is completely silent. I can't imagina th






can achieve 1 MByte / Sec up to 12 MByte/Sec














Nathan McNulty

kony wrote:

On Wed, 14 Jul 2004 17:53:52 +0200, "Skybuck Flying"
wrote:


Hi,

Take these 3 concepts and then look at the implementation/performance:

( 3 Concepts for reading/writing with harddisks )

Concept 1:

Seek, Read, Read, Read, Read, Read, Etc

Results: FAST

Concept 2:

Seek, Write, Write, Write, Write, Write, Etc

Results: FAST

Concept 3:

Seek, Read, Write, Read, Write, Read, Write,

Results: SLOW ???

The pseudo code is like:

Seek( 0 ); // offset 0

For I:=0 to FileBlocks-1 do // number of 4KB blocks in file.
begin
if Random(2) = 0 then
Read( 4 KB )
Else
Write( 4 KB );
end;

The original concept is:

Do a seek once.

Then read or write a block of data. The head is automatically forwarded
to
the next block. So no extra seek is needed.

For concept 1 and concept 2 this works just fine and gives good
performance.

However concept 3 has very bad performance.

Is this a software issue ? ( Windows XP )

Is this a hardware issue ? ( Harddisk Read Head and Harddisk Write Head
can't work together like this and an extra seek is needed ? )

Or some sort of driver issue ? ( Harddisk driver / firmware issue ? )

Bye,
Skybuck.




I may not know the answer but feel an important question might be
"What is very bad performance?", compared to good performance...
numbers are our friends.

Could it simply be that you're switching back and forth with data
flow so the caching (on the drive) isn't effective?

Hmmm...

I examined the source code of my read and write benchmark programs

The read program reads the totaly file.

So it does

1 Seek

X reads until file is read.

The write program however uses a cache of 1 MB.

So it does

1 Seek

X writes until cache is written and then again

1 Seek

X writes until cache is written etc etc.

So that can explain why writing is slower.

So I will have to write some new benchmarks... which behave exactly the same
for reading and writing

Bye,
Skybuck.

"Nathan McNulty" wrote in message
...
1. Yes really. You cannot read and write at the same time. The command
has to be either read or write.

2. Hardware limitation that it cannot read and write at the same time.

Not really, a logical/design fact.

3. From what I have seen on my old harddrives I have taken apart and
what I have read, there is a tip on the head that prevents the magnetic
part from affecting nearby sectors. I was under the impression these do
not always stay in the same position but are required to move each time
a write process is initiated.

Not true for anything recent.

4. Again, the tips would have to move back to their resting position.

No tips.

Ron Reaugh wrote:

"Nathan McNulty" wrote in message
...

Actually, what is happening is that the heads on the platter need to
reconfigure themselves each time you read or write.


Not really.


It is a hardware
limitation.


What limitation is that exactly?


By reading, the head can stay in the same location, but
once it is told to write, the head has to position itself to change the
magnetism of the sector of the platter it is on.


That's false.


Then once you request
to read again, the head has to move again.


Not necessarily.

"Skybuck Flying" wrote in message
...

Concept 3 (Reading and Writing, with regular switches )

Oh my... now that's what I call bad performance !

The first 10 seconds it was 80 Kbyte/Sec... the next 20 seconds 160
Kbyte/sec.

Max was 200 KByte/sec.

So it seems switching from read to write to read to write requires extra
seeking etc...

Nope, calculate the theoretical rate for the data group to be transferred
at the rate of one per revolution. Each time you switch from read to write
you likely lose the rest of the current disk rotation.

Maximum seek time for my HD and most HD's is 20 milliseconds.

What is the relation of the location of each 4KB block of data? Are they
sequential in a file? Where is a write located compared to the preceeding
read?.

Average Seek time is around 10 to 15 milliseconds.

Seeking has to do with moving the head to a different track. Are you doing
random I/O of these blocks in a big file or to multiple files?

It sounds like you haven't well thought out or at least described what you
are doing.