is "Everything" doing some mining?

J. P. Gilliver (John) wrote:

I used to open "Everything" and then leave it running. It takes a few
seconds to open, then settles down; I can use it, and it finds things
amazingly fast. It is an excellent utility!

However, of late: some minutes after I've opened it and all has settled
down, I hear my fan spin up (it is normally idling), and I start Task
Manager to see what's using CPU - and I see Everything.exe is using 25%
CPU (four apparent cores - I think it's an i3). This applies though I
haven't typed anything new into the search box. Closing the search
window doesn't stop it; however, selecting Exit from the menu that
appears when I right-click on the tray icon does stop it, within a small
number of seconds.

It's consistent - meaning Exit-ing it _always_ stops the 25% and fan, so
I am definite that Everything is the cause; I can't say that running it
always starts the 25% [I just started it again while typing this post,
and it hasn't gone berserk yet - Task Manager shows it sitting there at
00 CPU. It also came up immediately, with the search box showing all
files, i. e. without the few seconds' wait I usually experience when I
start it, so it presumably has some cache somewhere or something.]

I just have a simple system - two partitions on one HD; in particular, I
_don't_ have any network drives.

You might ask what I'm _doing_ when it goes berserk: I can't say it is
always this, but usually downloading a video file or two: I tend to have
Everything open as I use it to see if I've already got a particular file
before starting the download. But once E. has started its berserking,
even if I don't download any more, it doesn't stop 25%ing after the
current download has completed.

If voidtools _are_ doing some mining, I probably wouldn't mind, but I've
just looked at the website - there's no mention of such. Nor anything in
the FAQ about "why is Everything using so much CPU" or similar question.

If not (and I really don't think it is), I'm very puzzled about what it
_is_ doing! I did ask this here before, and I think there were
suggestions about doing a re-index; but I can't see why it should do
that continuously, especially when I'm not doing anything.

[FWIW, for the several minutes since I restarted it a few paragraphs
ago, it _hasn't_ gone above 00 in Task Manager. Maybe that's what I'll
have to do in futu start it, stop it when it goes berserk, then start
it again. But it seems decidedly odd!]

Its immediate search results require previous searching. Like Windows
own indexing, Search Everything has to build its own index. I suspect
it occasionally performs an update by scanning the drives to detect
anything that changed that its service might've not captured at the time
of change.

Unless I missed it, I did not see an option of when to reindex in the
background.

They have their own forums. Alas, unless you login you cannot search
their forums on inquiries about background indexing.

If you are concerned about data mining, use a network monitor, like
wireshark, to check if the program is phoning home. Make sure to
disable its auto-update check to eliminate that network traffic. If you
don't want all the history of traffic that wireshark affords along with
the complexity of that tool, you could use SysInternals' TCPview to
check what connections are defined at the time you notice the large
increase in CPU usage to see if there is any network traffic from the
everything.exe process(es).

Presumably you are using Search Everything in its default or
install-time setup. If you altered its configuration, like to enable
its HTTPS server to share its search database, then perhaps one of your
other hosts is querying that HTTP server. While you mentioned a higher
CPU usage, you also mentioned concerned about data mining and that would
require network traffic.

J. P. Gilliver (John) wrote:
In message , Mayayana
writes:
[Everything]
I'm surprised it needs to index if it doesn't do
content search. Can it actually be faster to search
its own list than to search the file system?

I think maybe it _does_ search the file system; certainly, when you are
typing in the part filename you are looking for, it seems to amend the
list it's presenting to you with each character you type, even at normal
typing speed.

Go on, give it a try - you can always uninstall it. (And I agree,
Everything is a confusing name!)

"Everything.exe" indexes in two stages, if it is starting
from scratch.

It reads the $MFT and parses it. This gives a list
of file names, but not their size or creation date.
This might take two seconds. The voidtools designers
really should have stopped at this point.

It's the second phase which is more expensive. If they
want to add dates and sizes and so on, file details,
that requires "walking the tree".

Finally, at some point, the "list" needs sorting.
Perhaps as a means of reducing search time on actual
searches (binary probing?).

None of these activities should particularly leave
the CPU railed, forever... Sorting a list takes
time. But not infinite time.

*******

Later, if a new file is created, it is added to the
NTFS USN Journal. "Everything.exe" hears of this, the item
is added to the list (insertion sort maybe), the size
and date noted. Adding a single file to the existing
index should not be particularly expensive. Only
the first scan of the day should be expensive (assuming
it regenerates the list for fun, once a day).

*******

How does Everything.exe handle Junction Points ?

Does it use a timer to "shut down" aberrant behavior ?

On newer OSes and the C: drive, a lot more care must
be exercised to avoid the "usual traps". A Junction Point
can cause looping recursion until the attempt hits a "path too long"
error and the software in question moves to the next file
or directory. This could potentially take slightly longer
on file systems where the 64K path length has been
enabled (Win10? Optional?).

Another "trap" for file traversing software, is to step
into a certain crypto directory, attempt to read a
certain file... and discover it's a named pipe and it
blocks on input and the visiting program stops in its
tracks. Everything.exe should not fall for that, because
Everything.exe does not "read" files, it only stats() them.
Programs like hashdeep have to be aware of things like
that, and the hashdeep command line is filled with various
letter options for all the "traps" Windows can throw at it.

*******

Even ProcMon can't help us in cases like this.

If Everything.exe was "scanning" the file system, then
ProcMon logs all the file system calls. That activity is
visible.

But if someone puts a "list" in memory and does QuickSort
on it, that's a CPU/memory bound activity, creating
zero entries in ProcMon. So we can't know what it is doing.
When an activity carries out no system calls, we are left
in the dark. Now, it's time for the logic analyzer and
"logging every address".

And that stopped being particularly feasible, a long time ago.
I used to have a $1500 sampling head for a $50K logic analyzer
to make this possible, and those *were* the good old days.

The interface, if still present on modern processors, is likely
to slow down execution so much, as to "disturb" whatever it is
you're trying to study. (At one time, we simply traced every address
and data pin. And "back-filled" using disassembly and software
trace analysis. Modern processors have a "thin" debug interface,
which, while it works, it would take a lot of cycles to
trace a single instruction. And if you had a 28 core processor,
you'd have 28 times as much of that to do.)

Paul

micky wrote:
In alt.windows7.general, on Sat, 16 Mar 2019 16:07:29 +0000, "J. P.
Gilliver (John)" wrote:

They're different purposes: Everything works on filenames, Agent Ransack
on file contents. I don't think Everything - because of the way it
works, which I don't understand, but it's something to do with NTFS, I
think - _does_ hammer the disc. I use it mostly - but not exclusively -
when I want to ask myself "have I already downloaded a file with x
[often a serial number] in its name".

I have Ransack but don't it because I really don't look for text in a
file. (But it was good to learn what the word ransack meant. I thought
it just meant to tear everyhing up. I didn't know it meant for the
purpose of finding something.

One should read the help file or whatever for Everything. It is capable
of a lot. For example, if there were a file with the same anme as
John's email address above, C:\ jp 55 uk would be a way to search for
it. It's more powerful than that, but I can't recall all that it can
do.

Not only does Agent Ransack search the content of files,
but, it will engage all your CPU cores while doing it.
If you do a content search on an SSD, it can keep the
bus interface on the SSD very busy.

Agent Ransack does nothing in advance. You pay the same price
for each search. The only "improvement" from one search to
another, comes from the System Read Cache, which improves
the I/O rates on repeated searches you might carry out.

Everything.exe caches filenames, dates, sizes. In the index
file. It does not index content, or read file content that
I know of.

Windows Search indexes everything, content, filename,
date, size, and keeps it in a gigabyte-sized database.
It only does content indexing on things it understands.
Text files are easy. Many other file types defy
analysis (try and find a text string in a movie file).
File types with "providers", it is the "provider"
that pulls the text string out of the text file.
Windows Search doesn't "inherently" know how to
carry out some sort of forensic exam. It needs
a lot of help from other pieces of software
(i.e. a piece of software "per type").

Paul

On Sat, 16 Mar 2019 16:18:47 +0000, "J. P. Gilliver (John)"
wrote:

In message , Ken Blake
writes:
On Sat, 16 Mar 2019 08:43:26 -0700, Ken Blake
wrote:

On Sat, 16 Mar 2019 13:51:20 +0000, "J. P. Gilliver (John)"
wrote:

I used to open "Everything" and then leave it running. It takes a few
seconds to open, then settles down; I can use it, and it finds things
amazingly fast. It is an excellent utility!


Ditto to all of the above. I'll just add that I also use Agent Ransack
to find files by content when I need to.

I have it, but rarely use it. But yes, it's the go-to for content search


Like you, I use it much less often than Everything. Still, it's good
to have it there.


However, of late: some minutes after I've opened it and all has settled
down, I hear my fan spin up (it is normally idling), and I start Task
Manager to see what's using CPU - and I see Everything.exe is using 25%
CPU (four apparent cores - I think it's an i3). This applies though I
haven't typed anything new into the search box.


Two points:

It's probably indexing new files.

Weird. In the two hours since my previous post, it hasn't gone berserk
at all - certainly the fan hasn't spun up beyond idle, and when I look
at it in Task Manager, it's still at 00.

25% isn't a big deal if nothing less is using much. If you're not


Ugh! That should be "nothing else" not "nothing less ."

(I didn't notice it!)

having a performance problem, I wouldn't worry about it; it will
probably go down again shortly.

Well, I wouldn't think so either, but it _does_ bring the fan up from
idle, which is distracting noise-wise -


Buy a quiter fan. g


and, if Toshiba's designers
think it needs that to happen, I presume it's heating up the processor
at least somewhat.

OK, but somewhat is very different from enough to cause a problem.


I don't notice any performance hit (but then most of


Then I wouldn't worry about it.


the time I'm not doing processor-intensive things anyway). I've tried
leaving it for several minutes, and it _doesn't_ go back down once it's
started doing it. What do you mean by "indexing new files"? I don't
_think_ I've usually added many since last time it happened; even if
that includes all the ones the OS is adding/deleting all the time, I
can't see it would be _that_ much work. Especially as, when I start
Everything from cold after a restart, it _does_ take a few tens of
seconds before filling the search window, so I assume it's doing some
indexing then - but, only for a few tens of seconds, and it does settle
down. I can't think what it could be doing indefinitely, or why it
should start sometime _after_ being started.

Closing the search
window doesn't stop it; however, selecting Exit from the menu that
appears when I right-click on the tray icon does stop it, within a small
number of seconds.

It's consistent - meaning Exit-ing it _always_ stops the 25% and fan, so
I am definite that Everything is the cause; I can't say that running it
always starts the 25% [I just started it again while typing this post,
and it hasn't gone berserk yet - Task Manager shows it sitting there at
00 CPU. It also came up immediately, with the search box showing all
files, i. e. without the few seconds' wait I usually experience when I
start it, so it presumably has some cache somewhere or something.]

I just have a simple system - two partitions on one HD; in particular, I
_don't_ have any network drives.

You might ask what I'm _doing_ when it goes berserk: I can't say it is
always this, but usually downloading a video file or two: I tend to have
Everything open as I use it to see if I've already got a particular file
before starting the download. But once E. has started its berserking,
even if I don't download any more, it doesn't stop 25%ing after the
current download has completed.

If voidtools _are_ doing some mining, I probably wouldn't mind, but I've
just looked at the website - there's no mention of such. Nor anything in
the FAQ about "why is Everything using so much CPU" or similar question.

If not (and I really don't think it is), I'm very puzzled about what it
_is_ doing! I did ask this here before, and I think there were
suggestions about doing a re-index; but I can't see why it should do
that continuously, especially when I'm not doing anything.

[FWIW, for the several minutes since I restarted it a few paragraphs
ago, it _hasn't_ gone above 00 in Task Manager. Maybe that's what I'll
have to do in futu start it, stop it when it goes berserk, then start
it again. But it seems decidedly odd!]

On Sat, 16 Mar 2019 16:07:29 +0000, "J. P. Gilliver (John)"
wrote:

In message , Mayayana
writes:
"J. P. Gilliver (John)" wrote

|I used to open "Everything" and then leave it running. It takes a few
| seconds to open, then settles down; I can use it, and it finds things
| amazingly fast. It is an excellent utility!
|
| However, of late: some minutes after I've opened it and all has settled
| down, I hear my fan spin up (it is normally idling), and I start Task
| Manager to see what's using CPU - and I see Everything.exe is using 25%
| CPU

I don't know about mining, but it does index. You should
be able to check the former by just blocking it from going
out.

You're right - I'd forgotten I have an indication (both graphical and
audible) (both directions) of net traffic, and that's silent, so it
isn't mining. I didn't think it was, really. (Though would mining
involve a lot of net traffic, or just a bit to fetch and a bit to return
the results?)

I've never tried Everything. I like Agent Ransack. Extremely
fast with no indexing. But if you have "a lot a lot" of stuff and
do a lot of searching, maybe indexing makes sense. For me,
I usually know pretty much where things are. I'm more apt
to do a search like finding which of 30 files in a folder has the
line of text I remember from an article I'm trying to find. Given
that, I think of indexing as wasteful wear and tear on disks.

They're different purposes: Everything works on filenames, Agent Ransack
on file contents.


No. Everything works on file names. Agent Ransack works on file
contents *or* file names. Agent Ransack is obviously better for
contents, since Everything doesn't do that, but Everything is *much*
faster (as I'm sure you know) for file names.

On Sat, 16 Mar 2019 19:49:00 +0000, "J. P. Gilliver (John)"
wrote:

In message , Mayayana
writes:
[Everything]
I'm surprised it needs to index if it doesn't do
content search. Can it actually be faster to search
its own list than to search the file system?

I think maybe it _does_ search the file system; certainly, when you are
typing in the part filename you are looking for, it seems to amend the
list it's presenting to you with each character you type, even at normal
typing speed.

Go on, give it a try - you can always uninstall it. (And I agree,
Everything is a confusing name!)


Yes. I usually call it "Search Everything."

On Sat, 16 Mar 2019 13:54:51 -0600, "Bill in Co"
surly_curmudgeon@earthlink wrote:

J. P. Gilliver (John) wrote:
In message , Mayayana
writes:
"J. P. Gilliver (John)" wrote

I used to open "Everything" and then leave it running. It takes a few
seconds to open, then settles down; I can use it, and it finds things
amazingly fast. It is an excellent utility!

However, of late: some minutes after I've opened it and all has settled
down, I hear my fan spin up (it is normally idling), and I start Task
Manager to see what's using CPU - and I see Everything.exe is using 25%
CPU

I don't know about mining, but it does index. You should
be able to check the former by just blocking it from going
out.

You're right - I'd forgotten I have an indication (both graphical and
audible) (both directions) of net traffic, and that's silent, so it
isn't mining. I didn't think it was, really. (Though would mining
involve a lot of net traffic, or just a bit to fetch and a bit to return
the results?)

I've never tried Everything. I like Agent Ransack. Extremely
fast with no indexing. But if you have "a lot a lot" of stuff and
do a lot of searching, maybe indexing makes sense. For me,
I usually know pretty much where things are. I'm more apt
to do a search like finding which of 30 files in a folder has the
line of text I remember from an article I'm trying to find. Given
that, I think of indexing as wasteful wear and tear on disks.

They're different purposes: Everything works on filenames, Agent Ransack
on file contents. I don't think Everything - because of the way it
works, which I don't understand, but it's something to do with NTFS, I
think - _does_ hammer the disc. I use it mostly - but not exclusively -
when I want to ask myself "have I already downloaded a file with x
[often a serial number] in its name".

You don't need to use Agent Ransack to look for text in a file. It works
great for just finding files by file name or file date, and without needing
indexing. So it seems to me they can serve similar purposes, except that I
don't want indexing, and if that's required for Everything, I myself
wouldn't use it. The only exception to this I can see is if I had a HUGE
database that needed to be searched frequently.



Your choice of course, but let me ask what you have against indexing.
It works extremely well for Everything.

"Bill in Co" surly_curmudgeon@earthlink wrote

| I use an older version of FileLocator Pro, the big brother to Agent
Ransack,
| which has a lot more options, like excluding directories from searches,
| which I find very advantageous. But it's not free, and unfortunately, has
| gotten a bit pricey over the years. But the option to exclude directories
| from searches greatly speeds up finding things, especially since I'm not
| using any indexing, by choice.
|

AR lets you start at any level/location. Maybe
you mean doing something like excluding System32
when searching Windows?

I use Search "Everything" Version 1.4.1.877 (x64) which I
downloaded about 6 months ago to update a much earlier version.
It has an "Advanced Search" that will search content and many
more things. --We should all get the latest version [;-)]. But
lately, I too have been noticing Search "Everything" interfering
with Windows 7 Explorer after using MSIE-11 while Search
"Everything" is active. Something causes Windows Explorer
windows to automatically refresh every 2 seconds or so after
this. To stop this behavior, I have to Log Off or Restart my
(desktop) computer. I have not checked what is going on in Task
Manager, though. I wonder if this is related to your issue, as
you say that it happens after you view a webpage.

Mayayana wrote:
"Bill in Co" surly_curmudgeon@earthlink wrote

I use an older version of FileLocator Pro, the big brother to Agent
Ransack, which has a lot more options, like excluding directories from
searches, which I find very advantageous. But it's not free, and
unfortunately, has gotten a bit pricey over the years. But the option
to exclude directories from searches greatly speeds up finding things,
especially since I'm not using any indexing, by choice.


AR lets you start at any level/location. Maybe
you mean doing something like excluding System32
when searching Windows?

Exactly. And even more than that!

Ken Blake wrote:
On Sat, 16 Mar 2019 13:54:51 -0600, "Bill in Co"
surly_curmudgeon@earthlink wrote:

J. P. Gilliver (John) wrote:
In message , Mayayana
writes:
"J. P. Gilliver (John)" wrote

I used to open "Everything" and then leave it running. It takes a few
seconds to open, then settles down; I can use it, and it finds things
amazingly fast. It is an excellent utility!

However, of late: some minutes after I've opened it and all has
settled down, I hear my fan spin up (it is normally idling), and I
start Task Manager to see what's using CPU - and I see Everything.exe
is using 25% CPU

I don't know about mining, but it does index. You should
be able to check the former by just blocking it from going
out.

You're right - I'd forgotten I have an indication (both graphical and
audible) (both directions) of net traffic, and that's silent, so it
isn't mining. I didn't think it was, really. (Though would mining
involve a lot of net traffic, or just a bit to fetch and a bit to return
the results?)

I've never tried Everything. I like Agent Ransack. Extremely
fast with no indexing. But if you have "a lot a lot" of stuff and
do a lot of searching, maybe indexing makes sense. For me,
I usually know pretty much where things are. I'm more apt
to do a search like finding which of 30 files in a folder has the
line of text I remember from an article I'm trying to find. Given
that, I think of indexing as wasteful wear and tear on disks.

They're different purposes: Everything works on filenames, Agent Ransack
on file contents. I don't think Everything - because of the way it
works, which I don't understand, but it's something to do with NTFS, I
think - _does_ hammer the disc. I use it mostly - but not exclusively -
when I want to ask myself "have I already downloaded a file with x
[often a serial number] in its name".

You don't need to use Agent Ransack to look for text in a file. It works
great for just finding files by file name or file date, and without
needing indexing. So it seems to me they can serve similar purposes,
except that I don't want indexing, and if that's required for
Everything, I myself wouldn't use it. The only exception to this I can
see is if I had a HUGE database that needed to be searched frequently.



Your choice of course, but let me ask what you have against indexing.
It works extremely well for Everything.

For me it's a general issue with indexing. The indexes may or may not be up
to date, for example, after adding or modifying or removing some files, but
without indexing, what you see is what you get, and there is no waiting for
an index to rebuilt each time. The downside is it takes longer to find
things, admitely. But I can wait for that. Even more especially if I use a
program like FileLocator Pro, which lets me exclude extraneous windows
directories in the search.

I also have indexing turned off on my system too. That's one of the first
things I did when I get a new computer, but in this case it may also be due
tothe potential for hogging some system resources, on occasion.

As I like to say, Less is (or can be) More, one of my general philosophies
of life. :-) (Somewhat of a Luddite, I guess. :-) Which is another
reason I'm still on XP).

"Bill in Co" surly_curmudgeon@earthlink wrote

|
| I also have indexing turned off on my system too. That's one of the first
| things I did when I get a new computer, but in this case it may also be
due
| tothe potential for hogging some system resources, on occasion.
|
| As I like to say, Less is (or can be) More, one of my general philosophies
| of life. :-) (Somewhat of a Luddite, I guess. :-) Which is another
| reason I'm still on XP).
|
You and I seem to agree about so many things. You
may be the most brilliant person I've ever met.

"Ken Blake" wrote

|
No. Everything works on file names. Agent Ransack works on file
contents *or* file names. Agent Ransack is obviously better for
contents, since Everything doesn't do that, but Everything is *much*
faster (as I'm sure you know) for file names.
|

----------------------------------------
https://www.voidtools.com/faq/#does_..._file_contents

Does Everything search file contents?

Yes, "Everything" can search file content with the content: search function.
File content is not indexed, searching content is slow.
----------------------------------------

It seems that the best way to think of Everything is
as a modified Explorer program, while AR is a search
program.

In message , Mayayana
writes:
"Ken Blake" wrote

|
No. Everything works on file names. Agent Ransack works on file
contents *or* file names. Agent Ransack is obviously better for
contents, since Everything doesn't do that, but Everything is *much*
faster (as I'm sure you know) for file names.
|

----------------------------------------
https://www.voidtools.com/faq/#does_..._file_contents

Does Everything search file contents?

Yes, "Everything" can search file content with the content: search function.
File content is not indexed, searching content is slow.
----------------------------------------

It seems that the best way to think of Everything is
as a modified Explorer program, while AR is a search
program.

I had never thought of Explorer as a search program. Probably because
(to me anyway) it's UI (for that function) is very non-intuitive.


--
J. P. Gilliver. UMRA: 1960/1985 MB++G()AL-IS-Ch++(p)Ar@T+H+Sh0!:`)DNAf

Try to learn something about everything and everything about something.
-Thomas Henry Huxley, biologist (1825-1895)

In message , Ken Blake
writes:
On Sat, 16 Mar 2019 16:18:47 +0000, "J. P. Gilliver (John)"
wrote:
[]
Well, I wouldn't think so either, but it _does_ bring the fan up from
idle, which is distracting noise-wise -


Buy a quiter fan. g

It's a laptop (-:

and, if Toshiba's designers
think it needs that to happen, I presume it's heating up the processor
at least somewhat.

OK, but somewhat is very different from enough to cause a problem.

I'm a bit paranoid about overheating, as I think that's what killed my
hard drive a couple of years ago.
[]
--
J. P. Gilliver. UMRA: 1960/1985 MB++G()AL-IS-Ch++(p)Ar@T+H+Sh0!:`)DNAf

The thing about smut is it harms no one and it's rarely cruel. Besides, it's a
gleeful rejection of the dreary and the "correct".
- Alison Graham, RT 2014/10/25-31