Why is search so brain dead these days?

On 2020-06-22 7:15 a.m., Ken Blake wrote:
On 6/21/2020 10:04 AM, philo wrote:
On 6/21/20 11:01 AM, Paul wrote:
philo wrote:

Thanks for the info.
As one who recently did a search that found close to nothing, I am
happy with the much improved results using the free version of
Agent Ransack.



Ransack : 52 hits in ten minutes

Â*From Explorer, after one hour , four hits...search nowhere near
complete



I just did a file search with Search Everything. It found all the
matching files on three physical drives in less than one second.

Then it builds an index, or there weren't very many files in the
directory to have to run through.



I use Agent Ransack only for finding text within files. For file name
searches, Search Everything is *much* faster.

Alan Baker wrote:
On 2020-06-21 12:48 p.m., Frank Slootweg wrote:
Alan Baker wrote:
On 2020-06-20 9:19 p.m., Paul wrote:
Alan Baker wrote:
On 2020-06-20 8:01 p.m., VanguardLH wrote:
Alan Baker wrote:

Mayayana wrote:

But can Spotlight find it all on Windows? I use Agent Ransack. It
finds text in files, file name segments, etc, at an amazing speed,
and I don't need indexing. Anything can quickly look up stored data
in a database, but the trouble is that such a program has to run
regularly to update its record. That's not necessary with Agent
Ransack. And best of all, AR can find the files on Windows. I don't
have any files on a Mac.

Ah, but the database is updated continuously.

Ah, you also have a reading comprehension defect, too.

In what particular?

Well, anything that "indexes", generally hooks the NTFS journal.

Except for other OSes that don't use NTFS.

*There*'s your "reading comprehension defect" for you!

Nope. Try again.

Wrong. Read on.

Mac OS uses a hook into the file system calls to trigger the metadata
server that file needs to be re-indexed. What's more, developers can add
specific importers for the files their software creates so that
Spotlight can index them seamlessly

And it all happens continuously

And guess what the NTFS file system has and does? It's above, in plain
sight. You only have to read and comprehend it. (FYI, NTFS was not the
first to have this functionality and neither was Apple.)

Apple's functionality is not based on having a journal.

It *is*. You yourself wrote (rewinding_to/repeating_from above):

[Rewind/repeat:]

Mac OS uses a hook into the file system calls to trigger the metadata
server that file needs to be re-indexed.

That's journaling. Whether the file system *keeps* the journaled
events - for *other* purposes, such as recovery - is another matter.

Both NTFS and MacOS' file system are ... file systems. No, I'm not
going to repeat the '...' bit. Paul explained that, but your needing to
brag about Apple (why?) made your brain lockup and 'miss' what was
explained.

Next.

The MythicSoftware tools, there are two of them. Agent Ransack
is free and brute force (it's intended as a teaser, to
get you to buy the other one). File Locator Pro is their for-sale product,
and as far as I know, it indexes. And because it indexes, it's
going to hook the journal (this doesn't seem that hard to do,
seeing as many have succeeded at it).

Oops! There's the 'secret' again! I hope you'll miss it this time
again

Again. I didn't miss anyting.

On the contrary, I picked up the erroneous claim that:

"because it indexes, it's going to hook the journal"

That's simply not true.

You use quotes, but you seem to refer to this - differently worded -
sentence:

[Rewind:]

Well, anything that "indexes", generally hooks the NTFS journal.

For the NTFS case (on Windows), it *is* true.

Everything.exe was the one with a lot of hopes riding on it.
Initially, it could index C: in about 2 seconds (having
never seen C: before). It could do this, because it read the $MFT
directly. They're not the first, nor the last, to try that.
[Agent Ransack doesn't read the $MFT, not that I can see.
It uses FindNextFile (brute force).]

However, after a few releases, Everything.exe got the usual complaints
about "why can't we see the file size in the listing?". That
is a more expensive option, requiring a directory level scan.
And it still does that today, so the time to index all of C:
rises from 2 seconds to maybe 20 seconds. Just so you can
have file sizes. Once the initial index is generated,
individual journal events like file-adds or file-deletes,
cause the index to be updated accordingly by the Everything
service.

Oops! There it was *again*! Shut the fsck (hint! hint!) up, Paul!

I haven't tried out too many of these things, and those
are some of the popular ones here. There are still people
trying to write them, for some reason.

Still not seeing how I failed of reading comprehension.

That's really *your* problem, isn't it!? You *could* be less pompous
and some kind soul might explain it to you, but there's little chance
of that, isn't there?

Certainly asking you to provide a real explanation isn't going to work...

So the consequences of your pompousity are my fault!?

Yes: if you hook up a Windows formatted disk to a Mac, you can get
spotlight to index it and it will perform its usual continuous indexing
process.

Unlikely, but theoretically possible, and totally irrelevant.

Unlikely, perhaps, but not possible in theory. It works.

I meant in the way it's done on Windows, i.e. using the NTFS journal.

You presented your 'Yes: ...' as a scenario in which MacOS could index
a Windows formatted disk (i.e. NTFS) in a way which would not be
possible on Windows itself. *That* scenario would be unlikely, because
1) it *is* of course possible on Windows and 2) it would need
NTFS-journal support in the foreign filesystem support part of MacOS. If
you think MacOS has this support, then give a URL of a document which
supports that.

In hindsight, I think some of the aggro is a result of the fact that
your response was to Paul's post - while Paul had no quarrel with you -
but was also and even mainly about VanguardLH's "Ah, you also have a
reading comprehension defect, too.".

On 6/22/2020 9:42 AM, philo wrote:
On 6/22/20 9:15 AM, Ken Blake wrote:
On 6/21/2020 10:04 AM, philo wrote:
On 6/21/20 11:01 AM, Paul wrote:
philo wrote:

Thanks for the info.
As one who recently did a search that found close to nothing, I am
happy with the much improved results using the free version of
Agent Ransack.



Ransack : 52 hits in ten minutes

Â*From Explorer, after one hour , four hits...search nowhere near
complete



I just did a file search with Search Everything. It found all the
matching files on three physical drives in less than one second.


I use Agent Ransack only for finding text within files. For file name
searches, Search Everything is *much* faster.




WOW. Going to try is ASAP


Thanks!


You're welcome. Glad to help.


--
Ken

On 6/20/2020 3:54 PM, Yousuf Khan wrote:
I'm referring mainly to Windows search, but this applies to a lot of
other search algorithms all over the place and on the Internet too. In
the olden days, search was very efficient and somewhat intuitive. For
example, let's say you try to do a search for "virtual" and expect you
might find something like VirtualBox, VirtualPC, whatever. But for some
reason, the current Windows search cannot find these. If you do a search
for the full name, then it may find them (hit and miss). In the old
days, these searches would find all instances where the string would
occur, even as part of a substring. It was very easy to do searches, and
you could even do multiple words to narrow down the searches. What has
gone wrong with search algorithms now?

Yousuf Khan


I suggest that NO ONE reply to Alan Baker or anyone else touting Apple
products. Such posts are OFF TOPIC in this newsgroup.

--
David E. Ross
http://www.rossde.com/

Trump claims he is the "law and order" President. Then, he
breaks the law by using copyrighted music at his rallies
without permission from the copyright owners.

On 6/22/20 12:49 PM, Ken Blake wrote:
On 6/22/2020 9:42 AM, philo wrote:
On 6/22/20 9:15 AM, Ken Blake wrote:
On 6/21/2020 10:04 AM, philo wrote:
On 6/21/20 11:01 AM, Paul wrote:
philo wrote:

Thanks for the info.
As one who recently did a search that found close to nothing, I
am happy with the much improved results using the free version of
Agent Ransack.



Ransack : 52 hits in ten minutes

Â*From Explorer, after one hour , four hits...search nowhere near
complete



I just did a file search with Search Everything. It found all the
matching files on three physical drives in less than one second.


I use Agent Ransack only for finding text within files. For file name
searches, Search Everything is *much* faster.




WOW. Going to try is ASAP


Thanks!


You're welcome. Glad to help.





I am very happy!

On 6/21/20 12:36 PM, David E. Ross wrote:
On 6/21/2020 9:01 AM, Paul wrote:
philo wrote:

Thanks for the info.
As one who recently did a search that found close to nothing, I am
happy with the much improved results using the free version of Agent
Ransack.



Ransack : 52 hits in ten minutes

From Explorer, after one hour , four hits...search nowhere near complete

I had to use a registry setting on a *clean* 2004 install, to
get the search to Index properly!

Behavior like that has apparently been around since the year 2015.

Paul


In the Programs and Features window, I selected "Turn Windowsfeatures on
or off". In the Windows Features window, I then uncheck the checkboxes
for Indexing Service and Windows Search. I also removed Search from
Start.

I only use Everything and Agent Ransack. I found that only disabling
the Indexing Service still resulted in some indexing as I opened some
folders. This would delay responing when I selected an item in such
folders.




Glad there are 3rd party utilities.

Amazing how Windows screwed this up

On 2020-06-22 10:48 a.m., Frank Slootweg wrote:
Alan Baker wrote:
On 2020-06-21 12:48 p.m., Frank Slootweg wrote:
Alan Baker wrote:
On 2020-06-20 9:19 p.m., Paul wrote:
Alan Baker wrote:
On 2020-06-20 8:01 p.m., VanguardLH wrote:
Alan Baker wrote:

Mayayana wrote:

But can Spotlight find it all on Windows? I use Agent Ransack. It
finds text in files, file name segments, etc, at an amazing speed,
and I don't need indexing. Anything can quickly look up stored data
in a database, but the trouble is that such a program has to run
regularly to update its record. That's not necessary with Agent
Ransack. And best of all, AR can find the files on Windows. I don't
have any files on a Mac.

Ah, but the database is updated continuously.

Ah, you also have a reading comprehension defect, too.

In what particular?

Well, anything that "indexes", generally hooks the NTFS journal.

Except for other OSes that don't use NTFS.

*There*'s your "reading comprehension defect" for you!

Nope. Try again.

Wrong. Read on.

Mac OS uses a hook into the file system calls to trigger the metadata
server that file needs to be re-indexed. What's more, developers can add
specific importers for the files their software creates so that
Spotlight can index them seamlessly

And it all happens continuously

And guess what the NTFS file system has and does? It's above, in plain
sight. You only have to read and comprehend it. (FYI, NTFS was not the
first to have this functionality and neither was Apple.)

Apple's functionality is not based on having a journal.

It *is*. You yourself wrote (rewinding_to/repeating_from above):

[Rewind/repeat:]

Mac OS uses a hook into the file system calls to trigger the metadata
server that file needs to be re-indexed.

That's journaling. Whether the file system *keeps* the journaled
events - for *other* purposes, such as recovery - is another matter.

No. It is NOT a journal if the events aren't kept.

You need to learn what "journal" means in this context:

'A journaling file system is a file system that keeps track of changes
not yet committed to the file system's main part by recording the
intentions of such changes in a data structure known as a "journal",
which is usually a circular log.'
https://en.wikipedia.org/wiki/Journaling_file_system


Both NTFS and MacOS' file system are ... file systems. No, I'm not
going to repeat the '...' bit. Paul explained that, but your needing to
brag about Apple (why?) made your brain lockup and 'miss' what was
explained.

Next.

The MythicSoftware tools, there are two of them. Agent Ransack
is free and brute force (it's intended as a teaser, to
get you to buy the other one). File Locator Pro is their for-sale product,
and as far as I know, it indexes. And because it indexes, it's
going to hook the journal (this doesn't seem that hard to do,
seeing as many have succeeded at it).

Oops! There's the 'secret' again! I hope you'll miss it this time
again

Again. I didn't miss anyting.

On the contrary, I picked up the erroneous claim that:

"because it indexes, it's going to hook the journal"

That's simply not true.

You use quotes, but you seem to refer to this - differently worded -
sentence:

[Rewind:]

Well, anything that "indexes", generally hooks the NTFS journal.

For the NTFS case (on Windows), it *is* true.

Nope. Indexing need not pay the slightest attention to the journal.
Doing so may not be as efficient, but there is no NECESSITY to use the
journal to create an index.

On 2020-06-22 11:29 a.m., David E. Ross wrote:
On 6/20/2020 3:54 PM, Yousuf Khan wrote:
I'm referring mainly to Windows search, but this applies to a lot of
other search algorithms all over the place and on the Internet too. In
the olden days, search was very efficient and somewhat intuitive. For
example, let's say you try to do a search for "virtual" and expect you
might find something like VirtualBox, VirtualPC, whatever. But for some
reason, the current Windows search cannot find these. If you do a search
for the full name, then it may find them (hit and miss). In the old
days, these searches would find all instances where the string would
occur, even as part of a substring. It was very easy to do searches, and
you could even do multiple words to narrow down the searches. What has
gone wrong with search algorithms now?

Yousuf Khan


I suggest that NO ONE reply to Alan Baker or anyone else touting Apple
products. Such posts are OFF TOPIC in this newsgroup.


You are certainly free to as you please.

:-)

On 6/21/20 3:08 PM, Yousuf Khan wrote:
On 6/21/2020 10:42 AM, philo wrote:
Ransack : 52 hits in ten minutes

Â*From Explorer, after one hour , four hits...search nowhere near complete

LOL! What exactly did you get it to search for?

Â*Â*Â*Â*Yousuf Khan


I just put in my daughter's name , got some photos and text documents

On 2020-06-22 12:30 p.m., philo wrote:
On 6/21/20 3:08 PM, Yousuf Khan wrote:
On 6/21/2020 10:42 AM, philo wrote:
Ransack : 52 hits in ten minutes

Â*From Explorer, after one hour , four hits...search nowhere near
complete

LOL! What exactly did you get it to search for?

Â*Â*Â*Â*Â*Yousuf Khan


I just put in my daughter's name , got some photos and text documents

All the hits on my sister-in-law's name on the Mac: 3 seconds for 1405 hits.

Just sayin'

:-)

Alan Baker wrote:
On 2020-06-22 7:15 a.m., Ken Blake wrote:
On 6/21/2020 10:04 AM, philo wrote:
On 6/21/20 11:01 AM, Paul wrote:
philo wrote:

Thanks for the info.
As one who recently did a search that found close to nothing, I am
happy with the much improved results using the free version of
Agent Ransack.

Ransack : 52 hits in ten minutes

*From Explorer, after one hour , four hits...search nowhere near
complete

I just did a file search with Search Everything. It found all the
matching files on three physical drives in less than one second.

Then it builds an index, or there weren't very many files in the
directory to have to run through.

Of course it builds an index. The point is, when, how and how often it
builds/updates the index(es).

As Paul explained [1], Search Everything uses the MFT of the NTFS file
system. It doesn't have to traverse directories by the brute force
FindNextFile method. So building the index from scratch is very fast.

Paul mentioned 2 seconds for C:. philo said 1 second for 3 drives
(which might not contain C:, hence be faster). They did not specifically
mention if that was building the index from scratch or for indexing the
changes since the last build, i.e. the previous run of Search Everyhing.

And it of course depends on which properties are indexed, whether or
not the USN journal is used, etc..

For me, indexing changes on C: and D: - i.e. when invoking Search
Everyhing - is so fast, I cannot time it, so well under a second. And
that was after weeks of non-use, i.e. non-(re)indexing.

[I index on Name, Path, Size, Date Modified and Date Recently Changed,
which I think are the defaults.]

I use Agent Ransack only for finding text within files. For file name
searches, Search Everything is *much* faster.

[1] In, amongst others;

Message-ID:

Alan Baker wrote:
On 2020-06-22 10:48 a.m., Frank Slootweg wrote:
Alan Baker wrote:
On 2020-06-21 12:48 p.m., Frank Slootweg wrote:
Alan Baker wrote:
On 2020-06-20 9:19 p.m., Paul wrote:
[...]
Again. I didn't miss anyting.

On the contrary, I picked up the erroneous claim that:

"because it indexes, it's going to hook the journal"

That's simply not true.

You use quotes, but you seem to refer to this - differently worded -
sentence:

[Rewind:]

Well, anything that "indexes", generally hooks the NTFS journal.

For the NTFS case (on Windows), it *is* true.

Nope. Indexing need not pay the slightest attention to the journal.
Doing so may not be as efficient, but there is no NECESSITY to use the
journal to create an index.

Sigh! Of course there is no *necessity*, otherwise one could not - for
example - index a FAT file system. But Paul said *"generally"* and I
said "For the NTFS case (on Windows), it *is* true.", i.e. I confirmed
what Paul said.

However in your made-up quote you removed both "generally" and "NTFS".
So you misrepresented the context and argued based on that
misrepresentation. Are you sure you aren't one of nospam's socks!? :-(

(AFAIC,) EOD.

On 2020-06-22 1:07 p.m., Frank Slootweg wrote:
Alan Baker wrote:
On 2020-06-22 10:48 a.m., Frank Slootweg wrote:
Alan Baker wrote:
On 2020-06-21 12:48 p.m., Frank Slootweg wrote:
Alan Baker wrote:
On 2020-06-20 9:19 p.m., Paul wrote:
[...]
Again. I didn't miss anyting.

On the contrary, I picked up the erroneous claim that:

"because it indexes, it's going to hook the journal"

That's simply not true.

You use quotes, but you seem to refer to this - differently worded -
sentence:

[Rewind:]

Well, anything that "indexes", generally hooks the NTFS journal.

For the NTFS case (on Windows), it *is* true.

Nope. Indexing need not pay the slightest attention to the journal.
Doing so may not be as efficient, but there is no NECESSITY to use the
journal to create an index.

Sigh! Of course there is no *necessity*, otherwise one could not - for
example - index a FAT file system. But Paul said *"generally"* and I
said "For the NTFS case (on Windows), it *is* true.", i.e. I confirmed
what Paul said.


But it isn't always true...

....or at least there is no reasonable way you can know if it is true in
every case of search applications for Windows.

However in your made-up quote you removed both "generally" and "NTFS".
So you misrepresented the context and argued based on that
misrepresentation. Are you sure you aren't one of nospam's socks!? :-(

Sorry, but I didn't make up that quote. I did snip it for brevity and
clarity, but the sentence didn't include "generally" or "NTFS":

'And because it indexes, it's going to hook the journal (this doesn't
seem that hard to do, seeing as many have succeeded at it). '

That insists on a correlation between indexing and the journal that
doesn't exist.

Read it as many times as you need until you understand that there is no
"because". You can have an index without hooking into the journal.
That's just a fact.

On 6/22/2020 3:03 PM, philo wrote:
Glad there are 3rd party utilities.

Amazing how Windows screwed this up

They employ hamsters to write code for them. The base API functions were
obviously written by macaques, but the secondary coding was definitely
done by hamsters. But not just experienced graduate hamsters, but
first-year comp-sci hampsters.

Yousuf Khan

On 6/22/2020 3:30 PM, philo wrote:
On 6/21/20 3:08 PM, Yousuf Khan wrote:
On 6/21/2020 10:42 AM, philo wrote:
Ransack : 52 hits in ten minutes

Â*From Explorer, after one hour , four hits...search nowhere near
complete

LOL! What exactly did you get it to search for?

Â*Â*Â*Â*Â*Yousuf Khan


I just put in my daughter's name , got some photos and text documents

So now that you've tried Search Everything too, how many of the files
did each one find and how fast? Talking about WSrc vs. Ransack vs. SE?
Did they all find everything but at different speeds?

Yousuf Khan