Why is search so brain dead these days?

On 6/21/2020 9:01 AM, Paul wrote:
philo wrote:

Thanks for the info.
As one who recently did a search that found close to nothing, I am
happy with the much improved results using the free version of Agent
Ransack.



Ransack : 52 hits in ten minutes

From Explorer, after one hour , four hits...search nowhere near complete

I had to use a registry setting on a *clean* 2004 install, to
get the search to Index properly!

Behavior like that has apparently been around since the year 2015.

Paul


In the Programs and Features window, I selected "Turn Windowsfeatures on
or off". In the Windows Features window, I then uncheck the checkboxes
for Indexing Service and Windows Search. I also removed Search from
Start.

I only use Everything and Agent Ransack. I found that only disabling
the Indexing Service still resulted in some indexing as I opened some
folders. This would delay responing when I selected an item in such
folders.

--
David E. Ross
http://www.rossde.com/

Who would you trust to provide accurate information about
COVID-19? Doctors who have studied viruses and treated
patients for years? Or a TV actor who tweets "cofefe"?

On 6/21/2020 9:39 AM, VanguardLH wrote:
Yousuf Khan wrote:

VanguardLH wrote:

Yousuf Khan wrote:

I'm referring mainly to Windows search, but this applies to a lot of
other search algorithms all over the place and on the Internet too.
In the olden days, search was very efficient and somewhat intuitive.
For example, let's say you try to do a search for "virtual" and
expect you might find something like VirtualBox, VirtualPC, whatever.
But for some reason, the current Windows search cannot find these.

Perhaps you did not configure Windows Search to include the C:\Program
Files and C:\Program Files (x86) folders (and their subfolders), or add
whatever folders contain the "virtual"-named files you expect to find.

No, everything is included.

By default, not everything is included under the system drive (normally
the C: drive), like the C:\Windows folder. Although I've included more
subfolders, I don't include everything under the C: drive.

If the files you were looking for were .exe filetypes, is the .exe
filetype included as one of those that Windows Search hunts for (the
Indexing Options - Advanced - Filetypes tab)? Are any locations
listed as exclusions?

When you view the Indexing Options dialog, does it say "Indexing in
progress"? How long does it take when you Rebuild the index database,
or does it hang? I've read where a corrupted file can hang the
indexing, so run "chkdsk c: /r". Did you include temp and download
folders in the target locations? Those could have changing files at the
time of indexing. Unfinished files are, by definition, considered
corrupted and can hang the indexing.

Do you use MS Outlook as your e-mail client? If so, is it configured to
incorporate Windows Search? If you use POP accounts, a PST file could
be corrupt, especially if the message store was allowed to exceed the
maximum size of the .pst file, so you have to run scanpst.exe to repair.
If you don't want to scan your e-mails using Windows Search, and the
regular (inbuilt) search within Outlook is sufficient for you to find
e-mails in its message stores, deselect Outlook as a target in Indexing
Options.

Did you check the state of the Windows Search service? Should be
startup = Automatic and status = Running.

A lot of search links in Windows rely on using the Bing service. Users
have found they can get empty search results until they disable the Bing
integration with Windows Search. I disabled Bing integration soon after
I did a fresh install of Windows 10. I don't remember if I did the
regedit hack or used a tweaker.

HKEY_CURRENT_USER\Software\Microsoft\Windows\Curre ntVersion\Search
Data item: BingSearchEnabled (if not present, create DWORD 32bit)
Data value: 1 = enabled, 0 = disabled.

Windows Search will then only return local search results, and not
attempt to connect out to Microsoft's Bing service.

Windows Search has a troubleshooter (although I've found them to rarely
fix a problem): Settings - Update & Security - Troubleshoot, scroll
down to "Find and fix other problems" section, and select the "search
and indexing" option. For the problem, select "Files don't appear in
search results".

With "Hide protected operating system files" ENABLED, the Everything
search application lists 201,728 objects (folders and files) in
C:\Windows. This includes 5,471 .exe files, 296 .dat files, and 35,451
..dll files. It even shows the contents of C:\Windows\system32\boot.

--
David E. Ross
http://www.rossde.com/

Who would you trust to provide accurate information about
COVID-19? Doctors who have studied viruses and treated
patients for years? Or a TV actor who tweets "cofefe"?

Alan Baker wrote:
On 2020-06-21 7:22 a.m., Yousuf Khan wrote:
On 6/21/2020 9:28 AM, nospam wrote:
unix find is not fast, especially compared to something with an index.

Depends on how much you restricted its search parameters.

Yousuf Khan

No.. ...it does not.

The slowest part of an Index-based search on Windows,
is the thing displaying the results.

Explorer.exe is particularly bad as a file manager,
with a lot of activity to display a screen of entries.

The window in Everything.exe might be of a lighter
weight, and the datestamp on the file was put in the
database when the item was entered in the database.
So no additional post-search work has to be done.
(Everything.exe "trusts" its database.)

Using the programmatic interface to Windows Search,
I can return a result with 50000 files in about
3 seconds, stored in a text file. I don't know if
a Spotlight would cheat on a thing like that, and
used lazy evaluation if the user scrolls the display
box. That can be a smarter strategy at times. Explorer
just seems to brute force stuff like that, and it
can be a major impediment to getting the results.

This is an example of a script to do a search. This particular
one looks for several kinds of images, based on files with
metadata of HorizontalSize and VerticalSize. A text file does
not have properties like that, but a JPG might. I didn't do
a further characterization, to see what the overhead would be
for a search with more output. It should be a linear function,
but we don't know where it intersects the axis (what the latency
of a search with no results is).

**************** Helper script "query.ps1" ********************
# powershell -file query.ps1 -TREEDIR "'C:\'"

param([string]$TREEDIR="'C:\'")

$sql = "SELECT System.ItemFolderPathDisplay, `
System.ItemName, `
System.Image.HorizontalSize, `
System.Image.VerticalSize FROM SYSTEMINDEX `
WHERE System.Image.HorizontalSize0 AND `
System.Image.VerticalSize0 AND `
SCOPE=$TREEDIR"

$provider = "provider=search.collatordso;extended properties=’application=windows’;"

$connector = new-object system.data.oledb.oledbdataadapter -argument $sql, $provider

$dataset = new-object system.data.dataset

if ($connector.fill($dataset)) { $dataset.tables[0] | Export-CSV query.csv }
**************** end of Helper script "query.ps1" **************

Paul

Alan Baker wrote:
On 2020-06-20 9:19 p.m., Paul wrote:
Alan Baker wrote:
On 2020-06-20 8:01 p.m., VanguardLH wrote:
Alan Baker wrote:

Mayayana wrote:

But can Spotlight find it all on Windows? I use Agent Ransack. It
finds text in files, file name segments, etc, at an amazing speed,
and I don't need indexing. Anything can quickly look up stored data
in a database, but the trouble is that such a program has to run
regularly to update its record. That's not necessary with Agent
Ransack. And best of all, AR can find the files on Windows. I don't
have any files on a Mac.

Ah, but the database is updated continuously.

Ah, you also have a reading comprehension defect, too.

In what particular?

Well, anything that "indexes", generally hooks the NTFS journal.

Except for other OSes that don't use NTFS.

*There*'s your "reading comprehension defect" for you!

Mac OS uses a hook into the file system calls to trigger the metadata
server that file needs to be re-indexed. What's more, developers can add
specific importers for the files their software creates so that
Spotlight can index them seamlessly

And it all happens continuously

And guess what the NTFS file system has and does? It's above, in plain
sight. You only have to read and comprehend it. (FYI, NTFS was not the
first to have this functionality and neither was Apple.)

Both NTFS and MacOS' file system are ... file systems. No, I'm not
going to repeat the '...' bit. Paul explained that, but your needing to
brag about Apple (why?) made your brain lockup and 'miss' what was
explained.

Next.


The MythicSoftware tools, there are two of them. Agent Ransack
is free and brute force (it's intended as a teaser, to
get you to buy the other one). File Locator Pro is their for-sale product,
and as far as I know, it indexes. And because it indexes, it's
going to hook the journal (this doesn't seem that hard to do,
seeing as many have succeeded at it).

Oops! There's the 'secret' again! I hope you'll miss it this time
again.

Everything.exe was the one with a lot of hopes riding on it.
Initially, it could index C: in about 2 seconds (having
never seen C: before). It could do this, because it read the $MFT
directly. They're not the first, nor the last, to try that.
[Agent Ransack doesn't read the $MFT, not that I can see.
It uses FindNextFile (brute force).]

However, after a few releases, Everything.exe got the usual complaints
about "why can't we see the file size in the listing?". That
is a more expensive option, requiring a directory level scan.
And it still does that today, so the time to index all of C:
rises from 2 seconds to maybe 20 seconds. Just so you can
have file sizes. Once the initial index is generated,
individual journal events like file-adds or file-deletes,
cause the index to be updated accordingly by the Everything
service.

Oops! There it was *again*! Shut the fsck (hint! hint!) up, Paul!

I haven't tried out too many of these things, and those
are some of the popular ones here. There are still people
trying to write them, for some reason.

Still not seeing how I failed of reading comprehension.

That's really *your* problem, isn't it!? You *could* be less pompous
and some kind soul might explain it to you, but there's little chance
of that, isn't there?

Yes: if you hook up a Windows formatted disk to a Mac, you can get
spotlight to index it and it will perform its usual continuous indexing
process.

Unlikely, but theoretically possible, and totally irrelevant.

Alan Baker wrote:
On 2020-06-20 6:13 p.m., VanguardLH wrote:
Alan Baker , an obvious Linux/Mac
proselytizer posting in the wrong newsgroup, wrote:

Yousuf Khan wrote:

I'm referring mainly to Windows search, ...

I suggest you try out a modern Mac and its Spotlight facility.

That is not a solution. Does nothing to address the problem. Go
inhabit your Mac newsgroups and stop bothering those using a different
OS than your choice.

When you do something about your trolls infecting Mac newsgroups, I'll
stop posting here.

Are you for real!? 'our' trolls? What about 'your' trolls in the
Windows (and Android) groups? Aren't *you* one of 'them'? You sure *act*
like one.

Until then, I'll educate you about how much better it could be for you.

Giving information is fine. Acting like an obnoxious pompous prat, not
so much.

On 6/21/2020 10:42 AM, philo wrote:
Ransack : 52 hits in ten minutes

From Explorer, after one hour , four hits...search nowhere near complete

LOL! What exactly did you get it to search for?

Yousuf Khan

On 6/21/2020 10:34 AM, nospam wrote:
In article , Yousuf Khan
wrote:

unix find is not fast, especially compared to something with an index.

Depends on how much you restricted its search parameters.

nope. it's always going to be slower than something that uses an index.

Not in my experience. Agent Ransack without an index vs. Windows search
with an index, not even close. Philo's previous post just proved it,
he's running the same search between Agent Ransack and WinSearch and
Agent Ransack had 52 hits in 10 minutes, and WinSearch is still running
after several hours and has had just 4 hits.

Yousuf Khan

On 6/21/2020 11:12 AM, VanguardLH wrote:
Yousuf Khan wrote:

T wrote:

What do you mean "these days"?Â* Windows search has always stunk.Â* And
it is gettig worse.

That is my point. There used to search utilities under DOS that
worked much better than this.

You mean running the 'dir' command? You can still do that in a command
shell (cmd.exe). For example, you could run:

dir {parms} %temp%\dirlist.txt & notepad %temp%\dirlist.txt

No, no, that was a bit too primitive, there were various downloadable
utils available in those days to do searches across folders and even
across drives. I think one of those searches were even available in the
4DOS command processor, which was command.com replacement. So many years
ago, can't remember it all.

Yousuf Khan

On Sun, 21 Jun 2020 09:27:10 -0400, Yousuf Khan wrote:
I recall there were various find-like utilities for searching under DOS
that were just as simple, powerful, and fast. Now it's all messed up.

What's messed up? Those utilities haven't stopped working. For
instance, there's my own free GREP:

https://oakroadsystems.com/sharware/grep.htm

(A donation is requested if you use it and like it.)

The options can be a bit daunting at first, just because GREP can do
so much. But in addition to the full manual there's a shorter "GREP
101" version, both with many examples. There's also a tour that
illustrates some of the advanced features.


--
Stan Brown, Tehachapi, California, USA https://BrownMath.com/
https://OakRoadSystems.com/
Shikata ga nai...

On 6/21/2020 12:39 PM, VanguardLH wrote:
By default, not everything is included under the system drive (normally
the C: drive), like the C:\Windows folder. Although I've included more
subfolders, I don't include everything under the C: drive.

Don't worry, I know what I'm doing. When I said everything is included,
you can believe me.

Yousuf Khan

On 6/21/2020 12:44 PM, Alan Baker wrote:
Ummmm... ...no.

'The find utility recursively descends the directory tree for each path
listed, evaluating an expression (composed of the ``primaries'' and
``operands'' listed below) in terms of each file in the tree.'

There is no way a recursive directory search of an entire drive will be
faster than an indexed search.

Just keep reading the rest of the postings here on this thread, Windows'
indexed searches are just horribly broken. Philo for example, had run
one search in Agent Ransack it found 52 hits in 10 minutes. Same search
in Windows search is still running after several hours and has only
found 4 hits!

Yousuf Khan

"Alan Baker" wrote

| Simple fact:
|
| You've all been complaining about Window's search functionality.
|
| Mac OS has a search that works really, really well.
|

I'd be happy to complain about Macs, but this is
a Windows group and someone was asking about
search. Since we're not using Macs it doesn't
help to know what Mac search can do. But if you
want to buy a Mac and pay me to use it, I promise
I'll gush about how amazing all 5 programs are. And
those kiddie icons are just so adorable.

On 6/21/2020 12:44 PM, VanguardLH wrote:
Well, that means you used 3rd party search tools under DOS. Why are you
averse to using 3rd party search tools under Windows?

I've already said I use Agent Ransack, but that doesn't integrate into
Explorer quite as much as the Windows Search does. Also right at the
beginning I said this is not just a criticism of just Windows Search,
but just modern searchers in general. Sometimes, you just want to do a
search of just a few files within a single directory, as a filter, and
it's completely useless when the search doesn't find the files that you
absolutely know are in there already!

Yousuf Khan

In article , Yousuf Khan
wrote:

unix find is not fast, especially compared to something with an index.

Depends on how much you restricted its search parameters.

nope. it's always going to be slower than something that uses an index.

Not in my experience.

then your experience is very limited.

Agent Ransack without an index vs. Windows search
with an index, not even close. Philo's previous post just proved it,
he's running the same search between Agent Ransack and WinSearch and
Agent Ransack had 52 hits in 10 minutes, and WinSearch is still running
after several hours and has had just 4 hits.

two apps is not much experience.

it also means that windows search is is a very bad implementation of
indexing, not that the concept itself is flawed.

imagine how slow a google search would be if it had to recursively scan
every web site each time you did a search.

In article , Mayayana
wrote:


I'd be happy to complain about Macs,

except that your complaints about macs and anything else related to
apple are driven by blind hatred and ignorance than actual problems...

but this is
a Windows group and someone was asking about
search. Since we're not using Macs it doesn't
help to know what Mac search can do. But if you
want to buy a Mac and pay me to use it, I promise
I'll gush about how amazing all 5 programs are. And
those kiddie icons are just so adorable.

....such as that.