Nil wrote:


Window's built-in search indexing is, indeed slow. Everything's
indexing is immediate and virtually undetectable. Likewise, it's search
is instantaneous.

It's a content-indexer, where the button to disable
the content aspect, and just index filenames,
that button doesn't work! I think this is some
"humor" from the Microsoft staff, this button
to nowhere.

To read every file content and build an inverted index
takes around three hours.

If the Microsoft indexer just did filenames,
that would take about 15 seconds, for comparison.
But, they have no intention of doing that.

And everyone who deals with inverted indexes, seems
to use the same method.

1) Index is empty. The very first chunk of stuff to
be indexed, fills this index relatively rapidly.
2) At a "magic size", the indexer switches modes.
a) Create a temporary (empty) index file.
b) Index a section of the target.
c) Merge( temporary-index, main-index )
This step is God-awful slow.
Delete( temporary-index )
d) Repeat until entire target is processed.

And the progress slows down and runs at the
reduced rate, due to the merging that is going on.
Presumably the merging gets slower, the larger
the index gets. The largest index I've heard of
to date, was around 12GB (but it could easily
get bigger than that).

It's possible to make the Windows Indexer run faster,
by turning off the "Back off" feature, but it really
doesn't help much. The "Back off" feature reduces
Indexer activity when the mouse gets moved. Slightly
more CPU is used, if you step away from the computer
and let it finish the indexing (when the default
back-off behavior is enabled).

And I don't think you can turn off the indexer entirely.
Even if you stop the Service for it, it'll start itself
again. Some other agent steps in, and turns it on again.
It almost behaves like "malware" :-/ You can reduce
the list of things to be indexed to "nothing", but
the Indexer still runs, and it doesn't take zero time
either, when the list is zero length.

Paul