Oddness with chkdsk?

Win8.1, all "important" updates installed.

Over the past couple of months, the events log reports critical disc
errors (DR0 has a bad block). So, I've run chkdsk /r several times,
which "fixes" the problem until a few days later, when the same error
occurs again. The odd thing is that chkdsk identifies and "ropes off"
*exactly the same* locations on the disc, even the same number of kilobytes!

So... what's going on here? Why would Windows try to write to the same
bad sectors of the disc?

--
Best regards,

Neil

Neil wrote:
Win8.1, all "important" updates installed.

Over the past couple of months, the events log reports critical disc
errors (DR0 has a bad block). So, I've run chkdsk /r several times,
which "fixes" the problem until a few days later, when the same error
occurs again. The odd thing is that chkdsk identifies and "ropes off"
*exactly the same* locations on the disc, even the same number of
kilobytes!

So... what's going on here? Why would Windows try to write to the same
bad sectors of the disc?


http://superuser.com/questions/72120...isk-read-error

"Unfortunately this is a bug that affects Windows 8, Windows 8.1,
Server 2012 and Server 2012 R2. CHKDSK under those OS is unable
to populate the hidden $BadClus file, which contains a list of
the defective sectors found in the disk.

If you run CHKDSK under Windows 7 it will work correctly, and
it will mark those clusters as bad in the $BadClus file.

Let's hope Microsoft fixes this bug for Windows 10!
"

I don't have any disks in the necessary state to carry out
such an OS-by-OS test, so you'll have to try the other OSes
for help.

I guess this is one situation, where you need plenty of
OSes to test.

Don't forget, you can install some OSes for 30 days without
a license key, and all you need is media (and a spare hard drive).

And that would be long enough to do a (working) CHKDSK run.
While Win7 media cannot be naively downloaded from Microsoft
(user needs to type in a license key), someone made a URL
generator so you can actually get a working download link
from Microsoft.

https://www.heidoc.net/joomla/techno...download-links

The Heidoc tool, merely computes a download URL for you.
A button on the interface, copies the URL to the clipboard.
You then paste the link into your browser, verify it is a
Microsoft URL, and get your copy of the OS (as desired).

https://www.heidoc.net/joomla/images...downloader.jpg

HTH,
Paul

On 9/30/2016 11:04 PM, Paul wrote:
Neil wrote:
Win8.1, all "important" updates installed.

Over the past couple of months, the events log reports critical disc
errors (DR0 has a bad block). So, I've run chkdsk /r several times,
which "fixes" the problem until a few days later, when the same error
occurs again. The odd thing is that chkdsk identifies and "ropes off"
*exactly the same* locations on the disc, even the same number of
kilobytes!

So... what's going on here? Why would Windows try to write to the same
bad sectors of the disc?


http://superuser.com/questions/72120...isk-read-error


"Unfortunately this is a bug that affects Windows 8, Windows 8.1,
Server 2012 and Server 2012 R2. CHKDSK under those OS is unable
to populate the hidden $BadClus file, which contains a list of
the defective sectors found in the disk.

If you run CHKDSK under Windows 7 it will work correctly, and
it will mark those clusters as bad in the $BadClus file.

Let's hope Microsoft fixes this bug for Windows 10!
"

I don't have any disks in the necessary state to carry out
such an OS-by-OS test, so you'll have to try the other OSes
for help.

I guess this is one situation, where you need plenty of
OSes to test.

Don't forget, you can install some OSes for 30 days without
a license key, and all you need is media (and a spare hard drive).

And that would be long enough to do a (working) CHKDSK run.
While Win7 media cannot be naively downloaded from Microsoft
(user needs to type in a license key), someone made a URL
generator so you can actually get a working download link
from Microsoft.

https://www.heidoc.net/joomla/techno...download-links


The Heidoc tool, merely computes a download URL for you.
A button on the interface, copies the URL to the clipboard.
You then paste the link into your browser, verify it is a
Microsoft URL, and get your copy of the OS (as desired).

https://www.heidoc.net/joomla/images...downloader.jpg

HTH,
Paul

Thanks for the links, Paul. I wonder why it hasn't been addressed,
especially if it affects server 2012? It would seem a very trivial
matter to get chkdsk to populate the bad clusters file, certainly more
worthy of an "important" update than the "get Win10" nonsense that MS
spent so much energy on.

--
Best regards,

Neil

On Sat, 1 Oct 2016 08:39:04 -0400, Neil wrote:

Thanks for the links, Paul. I wonder why it hasn't been addressed,
especially if it affects server 2012? It would seem a very trivial
matter to get chkdsk to populate the bad clusters file, certainly more
worthy of an "important" update than the "get Win10" nonsense that MS
spent so much energy on.


These days, hardly anyone needs chkdsk to populate the bad clusters
file. Most all modern hard drives do automatic bad-block replacement
on the hard drive itself.

If you have a hard drive with automatic bad-block replacement, then
your hard drive might have run out of spare 'replacement' blocks.
This could be a sign that your hard drive has developed too many
surface errors and should be replaced.


--
Kind regards
Ralph
🦊

Neil wrote:

Thanks for the links, Paul. I wonder why it hasn't been addressed,
especially if it affects server 2012? It would seem a very trivial
matter to get chkdsk to populate the bad clusters file, certainly more
worthy of an "important" update than the "get Win10" nonsense that MS
spent so much energy on.

I'm not aware of any mechanism to replace $BADCLUS.

I don't know how they handle situations now, where the automatic
block substitution on the drive itself, runs out of blocks.
Normally, there would be a SMART error, but in cases where
the surface damage is limited to a small portion of the
disk, in fact SMART will claim the disk is "healthy". I
had that happen to a disk here - a 70GB section of the
disk, had no more spare blocks in it, and was slow as
molasses due to all the substitutions. And yet, no
parameter in SMART indicated anything was amiss. But
an HDTune read benchmark, immediately shows the 70GB swath of
damage. My first warning, was the OS "felt slow". No other
indicators showed up.

Paul

On 10/1/2016 2:59 PM, Ralph Fox wrote:
On Sat, 1 Oct 2016 08:39:04 -0400, Neil wrote:

Thanks for the links, Paul. I wonder why it hasn't been addressed,
especially if it affects server 2012? It would seem a very trivial
matter to get chkdsk to populate the bad clusters file, certainly
more worthy of an "important" update than the "get Win10" nonsense
that MS spent so much energy on.


These days, hardly anyone needs chkdsk to populate the bad clusters
file. Most all modern hard drives do automatic bad-block replacement
on the hard drive itself.

If you have a hard drive with automatic bad-block replacement, then
your hard drive might have run out of spare 'replacement' blocks.
This could be a sign that your hard drive has developed too many
surface errors and should be replaced.

Interesting, but I don't think it applies in this case. The HD has about
1/3TB free, and the bad clusters amount to about 400k.

--
Best regards,

Neil

On 10/1/2016 3:43 PM, Paul wrote:
Neil wrote:

Thanks for the links, Paul. I wonder why it hasn't been addressed,
especially if it affects server 2012? It would seem a very trivial
matter to get chkdsk to populate the bad clusters file, certainly more
worthy of an "important" update than the "get Win10" nonsense that MS
spent so much energy on.

I'm not aware of any mechanism to replace $BADCLUS.

I don't know how they handle situations now, where the automatic
block substitution on the drive itself, runs out of blocks.
Normally, there would be a SMART error, but in cases where
the surface damage is limited to a small portion of the
disk, in fact SMART will claim the disk is "healthy". I
had that happen to a disk here - a 70GB section of the
disk, had no more spare blocks in it, and was slow as
molasses due to all the substitutions. And yet, no
parameter in SMART indicated anything was amiss. But
an HDTune read benchmark, immediately shows the 70GB swath of
damage. My first warning, was the OS "felt slow". No other
indicators showed up.

Paul

The drive has about 400k in bad sectors, and quite a bit of free space,
so this is a problem I haven't even seen in DOS 1.x days.

https://whereismydata.wordpress.com/...s-what-is-the/

"The $BadClus is one of the 16 key NTFS metadata files. Its role is to
track sectors that a damaged/unable to be used on the drive. The Bad
Clus has a MFT record number of 8 and, in the MFT, it comes just below
$BitMap and $Boot."

It doesn't seem all that difficult to me to write the data to a file, or
for the OS to read that file before writing data to the disc. I don't
know how a "known" bug of this type could get past beta testing.

--
Best regards,

Neil

On Sat, 1 Oct 2016 16:04:02 -0400, Neil wrote:
On 10/1/2016 2:59 PM, Ralph Fox wrote:
On Sat, 1 Oct 2016 08:39:04 -0400, Neil wrote:

Thanks for the links, Paul. I wonder why it hasn't been addressed,
especially if it affects server 2012? It would seem a very trivial
matter to get chkdsk to populate the bad clusters file, certainly
more worthy of an "important" update than the "get Win10" nonsense
that MS spent so much energy on.


These days, hardly anyone needs chkdsk to populate the bad clusters
file. Most all modern hard drives do automatic bad-block replacement
on the hard drive itself.

If you have a hard drive with automatic bad-block replacement, then
your hard drive might have run out of spare 'replacement' blocks.
This could be a sign that your hard drive has developed too many
surface errors and should be replaced.

Interesting, but I don't think it applies in this case. The HD has about
1/3TB free, and the bad clusters amount to about 400k.


HD free space is irrelevant. The HD firmware has its own separate
reserved area for replacement bad blocks. That reserved area does not
appear in the free space which you see inn the OS filesystem.


A typical implementation is that the HD has several extra blocks at
the end of each track. The OS filesystem does not see extra blocks on
each track but the HD firmware can use them.


In short:
* Bad block replacement is managed by the HD firmware, not by the
OS filesystem.
* The free space you are talking about is managed by the OS filesystem,
not by the HD firmware.


--
Kind regards
Ralph
🦊

On 10/1/2016 4:55 PM, Ralph Fox wrote:
On Sat, 1 Oct 2016 16:04:02 -0400, Neil wrote:
On 10/1/2016 2:59 PM, Ralph Fox wrote:
On Sat, 1 Oct 2016 08:39:04 -0400, Neil wrote:

Thanks for the links, Paul. I wonder why it hasn't been addressed,
especially if it affects server 2012? It would seem a very trivial
matter to get chkdsk to populate the bad clusters file, certainly
more worthy of an "important" update than the "get Win10" nonsense
that MS spent so much energy on.


These days, hardly anyone needs chkdsk to populate the bad clusters
file. Most all modern hard drives do automatic bad-block replacement
on the hard drive itself.

If you have a hard drive with automatic bad-block replacement, then
your hard drive might have run out of spare 'replacement' blocks.
This could be a sign that your hard drive has developed too many
surface errors and should be replaced.

Interesting, but I don't think it applies in this case. The HD has about
1/3TB free, and the bad clusters amount to about 400k.


HD free space is irrelevant. The HD firmware has its own separate
reserved area for replacement bad blocks. That reserved area does not
appear in the free space which you see inn the OS filesystem.


A typical implementation is that the HD has several extra blocks at
the end of each track. The OS filesystem does not see extra blocks on
each track but the HD firmware can use them.


In short:
* Bad block replacement is managed by the HD firmware, not by the
OS filesystem.
* The free space you are talking about is managed by the OS filesystem,
not by the HD firmware.


If I understand what you've written, it implies a few things.

* The HD's block management does not report a specific error to the OS,
and therefore one can't assess the situation. There would be no way to
know whether the disc is actually bad!

* The OS does not check the HD firmware, or the HD firmware doesn't make
this info available to the OS (this would be very bad design in either
case, which makes me skeptical about this one).

* Presuming that chkdsk's test functions are valid since it reads/writes
to all sectors, several extra blocks at the end of even one track would
far exceed the bad block space as reported by chkdsk.

If any of this is correct, I'm appalled!

--
Best regards,

Neil

On 10/1/2016 7:08 PM, Ken1943 wrote:
On Sat, 1 Oct 2016 18:20:12 -0400, Neil
wrote:

On 10/1/2016 4:55 PM, Ralph Fox wrote:
On Sat, 1 Oct 2016 16:04:02 -0400, Neil wrote:
On 10/1/2016 2:59 PM, Ralph Fox wrote:
On Sat, 1 Oct 2016 08:39:04 -0400, Neil wrote:

Thanks for the links, Paul. I wonder why it hasn't been addressed,
especially if it affects server 2012? It would seem a very trivial
matter to get chkdsk to populate the bad clusters file, certainly
more worthy of an "important" update than the "get Win10" nonsense
that MS spent so much energy on.


These days, hardly anyone needs chkdsk to populate the bad clusters
file. Most all modern hard drives do automatic bad-block replacement
on the hard drive itself.

If you have a hard drive with automatic bad-block replacement, then
your hard drive might have run out of spare 'replacement' blocks.
This could be a sign that your hard drive has developed too many
surface errors and should be replaced.

Interesting, but I don't think it applies in this case. The HD has about
1/3TB free, and the bad clusters amount to about 400k.


HD free space is irrelevant. The HD firmware has its own separate
reserved area for replacement bad blocks. That reserved area does not
appear in the free space which you see inn the OS filesystem.


A typical implementation is that the HD has several extra blocks at
the end of each track. The OS filesystem does not see extra blocks on
each track but the HD firmware can use them.


In short:
* Bad block replacement is managed by the HD firmware, not by the
OS filesystem.
* The free space you are talking about is managed by the OS filesystem,
not by the HD firmware.


If I understand what you've written, it implies a few things.

* The HD's block management does not report a specific error to the OS,
and therefore one can't assess the situation. There would be no way to
know whether the disc is actually bad!

* The OS does not check the HD firmware, or the HD firmware doesn't make
this info available to the OS (this would be very bad design in either
case, which makes me skeptical about this one).

* Presuming that chkdsk's test functions are valid since it reads/writes
to all sectors, several extra blocks at the end of even one track would
far exceed the bad block space as reported by chkdsk.

If any of this is correct, I'm appalled!

This is why we have S.M.A.R.T programs if the program collects the
info correctly, the drive does it and some day I will figure out a way
to understand the numbers.
Defraggler has the information. Guess we have to write down the
numbers and check for changes over years.

???
Google didn't provide me with a useful definition for your use of
S.M.A.R.T., so I have no idea what you're referring to. Please clarify!

--
Best regards,

Neil

On Sat, 1 Oct 2016 18:20:12 -0400, Neil wrote:
On 10/1/2016 4:55 PM, Ralph Fox wrote:
On Sat, 1 Oct 2016 16:04:02 -0400, Neil wrote:
On 10/1/2016 2:59 PM, Ralph Fox wrote:
On Sat, 1 Oct 2016 08:39:04 -0400, Neil wrote:

Thanks for the links, Paul. I wonder why it hasn't been addressed,
especially if it affects server 2012? It would seem a very trivial
matter to get chkdsk to populate the bad clusters file, certainly
more worthy of an "important" update than the "get Win10" nonsense
that MS spent so much energy on.


These days, hardly anyone needs chkdsk to populate the bad clusters
file. Most all modern hard drives do automatic bad-block replacement
on the hard drive itself.

If you have a hard drive with automatic bad-block replacement, then
your hard drive might have run out of spare 'replacement' blocks.
This could be a sign that your hard drive has developed too many
surface errors and should be replaced.

Interesting, but I don't think it applies in this case. The HD has about
1/3TB free, and the bad clusters amount to about 400k.


HD free space is irrelevant. The HD firmware has its own separate
reserved area for replacement bad blocks. That reserved area does not
appear in the free space which you see inn the OS filesystem.


A typical implementation is that the HD has several extra blocks at
the end of each track. The OS filesystem does not see extra blocks on
each track but the HD firmware can use them.


In short:
* Bad block replacement is managed by the HD firmware, not by the
OS filesystem.
* The free space you are talking about is managed by the OS filesystem,
not by the HD firmware.


If I understand what you've written, it implies a few things.

* The HD's block management does not report a specific error to the OS,
and therefore one can't assess the situation. There would be no way to
know whether the disc is actually bad!


To assess whether a disk is going bad, there is S.M.A.R.T.
https://en.wikipedia.org/wiki/S.M.A.R.T%2E


All hard drives have several physical bad blocks on the surface.
The HD appears perfect when it leaves the factory because bad-block
replacement is already initialized.

It is only a problem when the HD starts developing many more bad
blocks.


* The OS does not check the HD firmware, or the HD firmware doesn't make
this info available to the OS (this would be very bad design in either
case, which makes me skeptical about this one).


The HD firmware _does_ make disk reliability information available.
That is what S.M.A.R.T. is for.

https://en.wikipedia.org/wiki/S.M.A.R.T%2E


* Presuming that chkdsk's test functions are valid since it reads/writes
to all sectors,

chkdsk only reports on what the HD makes visible to the OS.

You need manufacturer-specific software from the HD manufacturer
to go deeper.


several extra blocks at the end of even one track would
far exceed the bad block space as reported by chkdsk.

I think something in your implication needs clarifying.

Let's say there are two extra blocks at the end of one track.
And lets say one of the two is already used by factory-initialzed
bad block replacement.

So there is one extra block still available for bad-block
replacement on the one track.

If I understand you correctly, you are saying that one block far
exceeds the bad block space as reported by chkdsk.


--
Kind regards
Ralph
🦊

On 10/1/2016 7:53 PM, Ralph Fox wrote:
On Sat, 1 Oct 2016 18:20:12 -0400, Neil wrote:
On 10/1/2016 4:55 PM, Ralph Fox wrote:
On Sat, 1 Oct 2016 16:04:02 -0400, Neil wrote:
On 10/1/2016 2:59 PM, Ralph Fox wrote:
On Sat, 1 Oct 2016 08:39:04 -0400, Neil wrote:

Thanks for the links, Paul. I wonder why it hasn't been addressed,
especially if it affects server 2012? It would seem a very trivial
matter to get chkdsk to populate the bad clusters file, certainly
more worthy of an "important" update than the "get Win10" nonsense
that MS spent so much energy on.


These days, hardly anyone needs chkdsk to populate the bad clusters
file. Most all modern hard drives do automatic bad-block replacement
on the hard drive itself.

If you have a hard drive with automatic bad-block replacement, then
your hard drive might have run out of spare 'replacement' blocks.
This could be a sign that your hard drive has developed too many
surface errors and should be replaced.

Interesting, but I don't think it applies in this case. The HD has about
1/3TB free, and the bad clusters amount to about 400k.


HD free space is irrelevant. The HD firmware has its own separate
reserved area for replacement bad blocks. That reserved area does not
appear in the free space which you see inn the OS filesystem.


A typical implementation is that the HD has several extra blocks at
the end of each track. The OS filesystem does not see extra blocks on
each track but the HD firmware can use them.


In short:
* Bad block replacement is managed by the HD firmware, not by the
OS filesystem.
* The free space you are talking about is managed by the OS filesystem,
not by the HD firmware.


If I understand what you've written, it implies a few things.

* The HD's block management does not report a specific error to the OS,
and therefore one can't assess the situation. There would be no way to
know whether the disc is actually bad!


To assess whether a disk is going bad, there is S.M.A.R.T.
https://en.wikipedia.org/wiki/S.M.A.R.T%2E

Thanks for this link. I'm not big on "buzzwords", but I am familiar with
the principles involved.

My point, above, is that the HD block management should provide specific
error info to the OS. If the generic "DR0 has a bad block" is their idea
of specific info, it's not a good design. I would give it another
acronym; Disc Under Management By A Significantly Screwed process. ;-)

All hard drives have several physical bad blocks on the surface.
The HD appears perfect when it leaves the factory because bad-block
replacement is already initialized.

It is only a problem when the HD starts developing many more bad
blocks.

This is really not a new situation, and has been handled in the past by
writing the bad block info to an OS-accessible file. If it no longer
does this, I am skeptical that read/write operations would work very
reliably.

The links in Paul's reply state that this hasn't changed, but that under
Win8.1, chkdsk is unable to populate that file. This seems more
plausible to me as the cause of my problem based on the behaviors I've
observed.


* The OS does not check the HD firmware, or the HD firmware doesn't make
this info available to the OS (this would be very bad design in either
case, which makes me skeptical about this one).


The HD firmware _does_ make disk reliability information available.
That is what S.M.A.R.T. is for.

https://en.wikipedia.org/wiki/S.M.A.R.T%2E


* Presuming that chkdsk's test functions are valid since it reads/writes
to all sectors,

chkdsk only reports on what the HD makes visible to the OS.

You need manufacturer-specific software from the HD manufacturer
to go deeper.

several extra blocks at the end of even one track would
far exceed the bad block space as reported by chkdsk.

I think something in your implication needs clarifying.

Let's say there are two extra blocks at the end of one track.
And lets say one of the two is already used by factory-initialzed
bad block replacement.

So there is one extra block still available for bad-block
replacement on the one track.

If I understand you correctly, you are saying that one block far
exceeds the bad block space as reported by chkdsk.

Yes. There's nothing special about tracks; when the track is full for
whatever reason, the data is written to available blocks in another
track. All that is required for reliability is an indication of what
blocks to avoid due to read/write disc errors. Chkdsk is (but this
discussion implies "was") a low-level operation that identified bad
blocks so that the OS wouldn't write to them in the first place. It is
in addition to any HD self management functions, and I don't see why it
wouldn't still be a useful tool.

--
Best regards,

Neil