Why is search so brain dead these days?

On Sun, 21 Jun 2020 16:21:50 -0400, Yousuf Khan
wrote:

On 6/21/2020 12:44 PM, Alan Baker wrote:
Ummmm... ...no.

'The find utility recursively descends the directory tree for each path
listed, evaluating an expression (composed of the ``primaries'' and
``operands'' listed below) in terms of each file in the tree.'

There is no way a recursive directory search of an entire drive will be
faster than an indexed search.

Just keep reading the rest of the postings here on this thread, Windows'
indexed searches are just horribly broken. Philo for example, had run
one search in Agent Ransack it found 52 hits in 10 minutes. Same search
in Windows search is still running after several hours and has only
found 4 hits!

I'd like to see that experiment peer reviewed and duplicated before it gets
written in stone. Like, what kind of search (filename versus file contents
versus both) and across what type of storage and how is it connected, etc.

Yousuf Khan wrote:
On 6/21/2020 10:42 AM, philo wrote:
Ransack : 52 hits in ten minutes

From Explorer, after one hour , four hits...search nowhere near complete

LOL! What exactly did you get it to search for?

Yousuf Khan

It would have searched, whatever drive letter he
asked it to search.

What I found disturbing, on my "broken" Windows Search
on a fresh 2004 x64 install, was what I was seeing in
Process Monitor. I was seeing Explorer making a ton
of calls to the Registry, looking for filenames or
fragments of filenames. Which might have been why
my machine was pretty busy while the "search" was happening.

Since I reset the indexing, using the registry key,
it appears to have stopped doing whatever it was
previously doing.

It's pretty hard to believe this is a race condition,
where some things initializing out of the box, resulted
in that kind of functional failure. Explorer did not appear
to be traversing the file system, and the "green bar"
seemed related to File Explorer poking at a registry hive.

It wasn't "just" doing FindNextFile. The time spent, is
spent in some other futile exercise. Which I've now fixed.
It comes back quickly now.

Paul

Yousuf Khan wrote:
On 6/21/2020 12:44 PM, Alan Baker wrote:
Ummmm... ...no.

'The find utility recursively descends the directory tree for each
path listed, evaluating an expression (composed of the ``primaries''
and ``operands'' listed below) in terms of each file in the tree.'

There is no way a recursive directory search of an entire drive will
be faster than an indexed search.

Just keep reading the rest of the postings here on this thread, Windows'
indexed searches are just horribly broken. Philo for example, had run
one search in Agent Ransack it found 52 hits in 10 minutes. Same search
in Windows search is still running after several hours and has only
found 4 hits!

Yousuf Khan

But you can fix it.

I fixed mine!

Paul

Stan Brown wrote:
On Sun, 21 Jun 2020 09:27:10 -0400, Yousuf Khan wrote:
I recall there were various find-like utilities for searching under DOS
that were just as simple, powerful, and fast. Now it's all messed up.

What's messed up? Those utilities haven't stopped working. For
instance, there's my own free GREP:

https://oakroadsystems.com/sharware/grep.htm

(A donation is requested if you use it and like it.)

The options can be a bit daunting at first, just because GREP can do
so much. But in addition to the full manual there's a shorter "GREP
101" version, both with many examples. There's also a tour that
illustrates some of the advanced features.

http://gnuwin32.sourceforge.net/packages.html

There is a separate GREP, as well as a FileUtils
that has a whole bunch of different utilities.

*******

On Windows, there is native "findstr" as the "resident poor-mans grep".
It's not very good, but can be used in scripts for the
basics. That might be reserved for those who insist on using
only native content when they script.

Windows 10 Bash on WSL also has all those utilities, but
with some details about line endings that might matter.
Whereas the GNUWIN32 ones, generally do the right thing
with Windows line endings.

Paul

On Sun, 21 Jun 2020 18:08:54 -0400, Paul wrote:
Stan Brown wrote:
[quoted text muted]
The options can be a bit daunting at first, just because GREP can do
so much. But in addition to the full manual there's a shorter "GREP
101" version, both with many examples. There's also a tour that
illustrates some of the advanced features.

http://gnuwin32.sourceforge.net/packages.html

There is a separate GREP, as well as a FileUtils
that has a whole bunch of different utilities.

That's true, and I use GAWK very frequently. But I think you'll find
that my GREP has some options that are not present in the GNU port.

--
Stan Brown, Tehachapi, California, USA https://BrownMath.com/
https://OakRoadSystems.com/
Shikata ga nai...

On 6/21/2020 5:33 PM, Char Jackson wrote:
I'd like to see that experiment peer reviewed and duplicated before it gets
written in stone. Like, what kind of search (filename versus file contents
versus both) and across what type of storage and how is it connected, etc.

With Microsoft products, it's no longer anecdotal, it just gets added to
the existing database of findings.

Yousuf Khan

On Mon, 22 Jun 2020 00:50:01 -0400, Yousuf Khan
wrote:

On 6/21/2020 5:33 PM, Char Jackson wrote:
I'd like to see that experiment peer reviewed and duplicated before it gets
written in stone. Like, what kind of search (filename versus file contents
versus both) and across what type of storage and how is it connected, etc.

With Microsoft products, it's no longer anecdotal, it just gets added to
the existing database of findings.

I've never seen Agent Ransack run for 10 minutes. That's why I was asking.
My guess is that it was being asked to do a content search versus a
filename search.

On 2020-06-21 12:48 p.m., Frank Slootweg wrote:
Alan Baker wrote:
On 2020-06-20 9:19 p.m., Paul wrote:
Alan Baker wrote:
On 2020-06-20 8:01 p.m., VanguardLH wrote:
Alan Baker wrote:

Mayayana wrote:

But can Spotlight find it all on Windows? I use Agent Ransack. It
finds text in files, file name segments, etc, at an amazing speed,
and I don't need indexing. Anything can quickly look up stored data
in a database, but the trouble is that such a program has to run
regularly to update its record. That's not necessary with Agent
Ransack. And best of all, AR can find the files on Windows. I don't
have any files on a Mac.

Ah, but the database is updated continuously.

Ah, you also have a reading comprehension defect, too.

In what particular?

Well, anything that "indexes", generally hooks the NTFS journal.

Except for other OSes that don't use NTFS.

*There*'s your "reading comprehension defect" for you!

Nope. Try again.


Mac OS uses a hook into the file system calls to trigger the metadata
server that file needs to be re-indexed. What's more, developers can add
specific importers for the files their software creates so that
Spotlight can index them seamlessly

And it all happens continuously

And guess what the NTFS file system has and does? It's above, in plain
sight. You only have to read and comprehend it. (FYI, NTFS was not the
first to have this functionality and neither was Apple.)

Apple's functionality is not based on having a journal.


Both NTFS and MacOS' file system are ... file systems. No, I'm not
going to repeat the '...' bit. Paul explained that, but your needing to
brag about Apple (why?) made your brain lockup and 'miss' what was
explained.

Next.


The MythicSoftware tools, there are two of them. Agent Ransack
is free and brute force (it's intended as a teaser, to
get you to buy the other one). File Locator Pro is their for-sale product,
and as far as I know, it indexes. And because it indexes, it's
going to hook the journal (this doesn't seem that hard to do,
seeing as many have succeeded at it).

Oops! There's the 'secret' again! I hope you'll miss it this time
again


Again. I didn't miss anyting.

On the contrary, I picked up the erroneous claim that:

"because it indexes, it's going to hook the journal"

That's simply not true.


Everything.exe was the one with a lot of hopes riding on it.
Initially, it could index C: in about 2 seconds (having
never seen C: before). It could do this, because it read the $MFT
directly. They're not the first, nor the last, to try that.
[Agent Ransack doesn't read the $MFT, not that I can see.
It uses FindNextFile (brute force).]

However, after a few releases, Everything.exe got the usual complaints
about "why can't we see the file size in the listing?". That
is a more expensive option, requiring a directory level scan.
And it still does that today, so the time to index all of C:
rises from 2 seconds to maybe 20 seconds. Just so you can
have file sizes. Once the initial index is generated,
individual journal events like file-adds or file-deletes,
cause the index to be updated accordingly by the Everything
service.

Oops! There it was *again*! Shut the fsck (hint! hint!) up, Paul!

I haven't tried out too many of these things, and those
are some of the popular ones here. There are still people
trying to write them, for some reason.

Still not seeing how I failed of reading comprehension.

That's really *your* problem, isn't it!? You *could* be less pompous
and some kind soul might explain it to you, but there's little chance
of that, isn't there?

Certainly asking you to provide a real explanation isn't going to work...


Yes: if you hook up a Windows formatted disk to a Mac, you can get
spotlight to index it and it will perform its usual continuous indexing
process.

Unlikely, but theoretically possible, and totally irrelevant.

Unlikely, perhaps, but not possible in theory. It works.

On 2020-06-21 12:54 p.m., Frank Slootweg wrote:
Alan Baker wrote:
On 2020-06-20 6:13 p.m., VanguardLH wrote:
Alan Baker , an obvious Linux/Mac
proselytizer posting in the wrong newsgroup, wrote:

Yousuf Khan wrote:

I'm referring mainly to Windows search, ...

I suggest you try out a modern Mac and its Spotlight facility.

That is not a solution. Does nothing to address the problem. Go
inhabit your Mac newsgroups and stop bothering those using a different
OS than your choice.

When you do something about your trolls infecting Mac newsgroups, I'll
stop posting here.

Are you for real!? 'our' trolls? What about 'your' trolls in the
Windows (and Android) groups? Aren't *you* one of 'them'? You sure *act*
like one.

Until then, I'll educate you about how much better it could be for you.

Giving information is fine. Acting like an obnoxious pompous prat, not
so much.


I only came here because of a cross-poster.

I'm staying for the fun of it.

On 2020-06-21 1:24 p.m., Mayayana wrote:
"Alan Baker" wrote

| Simple fact:
|
| You've all been complaining about Window's search functionality.
|
| Mac OS has a search that works really, really well.
|

I'd be happy to complain about Macs, but this is
a Windows group and someone was asking about
search. Since we're not using Macs it doesn't
help to know what Mac search can do. But if you
want to buy a Mac and pay me to use it, I promise
I'll gush about how amazing all 5 programs are. And
those kiddie icons are just so adorable.



It helps to understand what can be done.

"Char Jackson" wrote in message
...
On Mon, 22 Jun 2020 00:50:01 -0400, Yousuf Khan
wrote:

On 6/21/2020 5:33 PM, Char Jackson wrote:
I'd like to see that experiment peer reviewed and duplicated before it
gets
written in stone. Like, what kind of search (filename versus file
contents
versus both) and across what type of storage and how is it connected,
etc.

With Microsoft products, it's no longer anecdotal, it just gets added to
the existing database of findings.

I've never seen Agent Ransack run for 10 minutes. That's why I was asking.
My guess is that it was being asked to do a content search versus a
filename search.

I use Agent Ransack to view my Freeware List files.
240+ text files totalling 3mb & the search takes a couple of seconds.
Give it my 'Documents' folder @ 19GB & I've closed the search, a quarter way
through, after an hour.
... Probably not the best place to store 6.4GB of drivers but all the rest
are text, Pdf & image files.

--
Regards
wasbit

On 6/21/2020 10:04 AM, philo wrote:
On 6/21/20 11:01 AM, Paul wrote:
philo wrote:

Thanks for the info.
As one who recently did a search that found close to nothing, I am
happy with the much improved results using the free version of Agent
Ransack.



Ransack : 52 hits in ten minutes

Â*From Explorer, after one hour , four hits...search nowhere near complete



I just did a file search with Search Everything. It found all the
matching files on three physical drives in less than one second.


I use Agent Ransack only for finding text within files. For file name
searches, Search Everything is *much* faster.

--
Ken

Paul on Sun, 21 Jun 2020 17:37:52 -0400 typed
in alt.windows7.general the following:
Yousuf Khan wrote:
On 6/21/2020 10:42 AM, philo wrote:
Ransack : 52 hits in ten minutes

From Explorer, after one hour , four hits...search nowhere near complete

LOL! What exactly did you get it to search for?

Yousuf Khan

It would have searched, whatever drive letter he
asked it to search.

What I found disturbing, on my "broken" Windows Search
on a fresh 2004 x64 install, was what I was seeing in
Process Monitor. I was seeing Explorer making a ton
of calls to the Registry, looking for filenames or
fragments of filenames. Which might have been why
my machine was pretty busy while the "search" was happening.

Since I reset the indexing, using the registry key,
it appears to have stopped doing whatever it was
previously doing.

It's pretty hard to believe this is a race condition,
where some things initializing out of the box, resulted
in that kind of functional failure. Explorer did not appear
to be traversing the file system, and the "green bar"
seemed related to File Explorer poking at a registry hive.

It wasn't "just" doing FindNextFile. The time spent, is
spent in some other futile exercise. Which I've now fixed.
It comes back quickly now.

Paul

This seems to be my experience opinion: that Windows is now
spending a lot of time and resources handling the overhead it has
created. If it was a human office, it has long reached the point
where most of the work is tracking and documenting the work which is
suppose to be done. "We need a meeting to find out why there has been
no progress."




--
pyotr filipivich
Next month's Panel: Graft - Boon or blessing?

On 6/22/20 9:15 AM, Ken Blake wrote:
On 6/21/2020 10:04 AM, philo wrote:
On 6/21/20 11:01 AM, Paul wrote:
philo wrote:

Thanks for the info.
As one who recently did a search that found close to nothing, I am
happy with the much improved results using the free version of
Agent Ransack.



Ransack : 52 hits in ten minutes

Â*From Explorer, after one hour , four hits...search nowhere near
complete



I just did a file search with Search Everything. It found all the
matching files on three physical drives in less than one second.


I use Agent Ransack only for finding text within files. For file name
searches, Search Everything is *much* faster.




WOW. Going to try is ASAP


Thanks!

On 6/22/20 11:42 AM, philo wrote:
On 6/22/20 9:15 AM, Ken Blake wrote:
On 6/21/2020 10:04 AM, philo wrote:
On 6/21/20 11:01 AM, Paul wrote:
philo wrote:

Thanks for the info.
As one who recently did a search that found close to nothing, I am
happy with the much improved results using the free version of
Agent Ransack.



Ransack : 52 hits in ten minutes

Â*From Explorer, after one hour , four hits...search nowhere near
complete



I just did a file search with Search Everything. It found all the
matching files on three physical drives in less than one second.


I use Agent Ransack only for finding text within files. For file name
searches, Search Everything is *much* faster.




WOW. Going to try is ASAP


Thanks!





Found even more files almost instantly


Just sent them a $10 donation