is "Everything" doing some mining?

I used to open "Everything" and then leave it running. It takes a few
seconds to open, then settles down; I can use it, and it finds things
amazingly fast. It is an excellent utility!

However, of late: some minutes after I've opened it and all has settled
down, I hear my fan spin up (it is normally idling), and I start Task
Manager to see what's using CPU - and I see Everything.exe is using 25%
CPU (four apparent cores - I think it's an i3). This applies though I
haven't typed anything new into the search box. Closing the search
window doesn't stop it; however, selecting Exit from the menu that
appears when I right-click on the tray icon does stop it, within a small
number of seconds.

It's consistent - meaning Exit-ing it _always_ stops the 25% and fan, so
I am definite that Everything is the cause; I can't say that running it
always starts the 25% [I just started it again while typing this post,
and it hasn't gone berserk yet - Task Manager shows it sitting there at
00 CPU. It also came up immediately, with the search box showing all
files, i. e. without the few seconds' wait I usually experience when I
start it, so it presumably has some cache somewhere or something.]

I just have a simple system - two partitions on one HD; in particular, I
_don't_ have any network drives.

You might ask what I'm _doing_ when it goes berserk: I can't say it is
always this, but usually downloading a video file or two: I tend to have
Everything open as I use it to see if I've already got a particular file
before starting the download. But once E. has started its berserking,
even if I don't download any more, it doesn't stop 25%ing after the
current download has completed.

If voidtools _are_ doing some mining, I probably wouldn't mind, but I've
just looked at the website - there's no mention of such. Nor anything in
the FAQ about "why is Everything using so much CPU" or similar question.

If not (and I really don't think it is), I'm very puzzled about what it
_is_ doing! I did ask this here before, and I think there were
suggestions about doing a re-index; but I can't see why it should do
that continuously, especially when I'm not doing anything.

[FWIW, for the several minutes since I restarted it a few paragraphs
ago, it _hasn't_ gone above 00 in Task Manager. Maybe that's what I'll
have to do in futu start it, stop it when it goes berserk, then start
it again. But it seems decidedly odd!]
--
J. P. Gilliver. UMRA: 1960/1985 MB++G()AL-IS-Ch++(p)Ar@T+H+Sh0!:`)DNAf

Radio 4 is one of the reasons being British is good. It's not a subset of
Britain - it's almost as if Britain is a subset of Radio 4. - Stephen Fry, in
Radio Times, 7-13 June, 2003.

"J. P. Gilliver (John)" wrote

|I used to open "Everything" and then leave it running. It takes a few
| seconds to open, then settles down; I can use it, and it finds things
| amazingly fast. It is an excellent utility!
|
| However, of late: some minutes after I've opened it and all has settled
| down, I hear my fan spin up (it is normally idling), and I start Task
| Manager to see what's using CPU - and I see Everything.exe is using 25%
| CPU

I don't know about mining, but it does index. You should
be able to check the former by just blocking it from going
out.

I've never tried Everything. I like Agent Ransack. Extremely
fast with no indexing. But if you have "a lot a lot" of stuff and
do a lot of searching, maybe indexing makes sense. For me,
I usually know pretty much where things are. I'm more apt
to do a search like finding which of 30 files in a folder has the
line of text I remember from an article I'm trying to find. Given
that, I think of indexing as wasteful wear and tear on disks.

In message , Mayayana
writes:
"J. P. Gilliver (John)" wrote

|I used to open "Everything" and then leave it running. It takes a few
| seconds to open, then settles down; I can use it, and it finds things
| amazingly fast. It is an excellent utility!
|
| However, of late: some minutes after I've opened it and all has settled
| down, I hear my fan spin up (it is normally idling), and I start Task
| Manager to see what's using CPU - and I see Everything.exe is using 25%
| CPU

I don't know about mining, but it does index. You should
be able to check the former by just blocking it from going
out.

You're right - I'd forgotten I have an indication (both graphical and
audible) (both directions) of net traffic, and that's silent, so it
isn't mining. I didn't think it was, really. (Though would mining
involve a lot of net traffic, or just a bit to fetch and a bit to return
the results?)

I've never tried Everything. I like Agent Ransack. Extremely
fast with no indexing. But if you have "a lot a lot" of stuff and
do a lot of searching, maybe indexing makes sense. For me,
I usually know pretty much where things are. I'm more apt
to do a search like finding which of 30 files in a folder has the
line of text I remember from an article I'm trying to find. Given
that, I think of indexing as wasteful wear and tear on disks.

They're different purposes: Everything works on filenames, Agent Ransack
on file contents. I don't think Everything - because of the way it
works, which I don't understand, but it's something to do with NTFS, I
think - _does_ hammer the disc. I use it mostly - but not exclusively -
when I want to ask myself "have I already downloaded a file with x
[often a serial number] in its name".

--
J. P. Gilliver. UMRA: 1960/1985 MB++G()AL-IS-Ch++(p)Ar@T+H+Sh0!:`)DNAf

WANTED, Dead AND Alive: Schrodinger's Cat

"J. P. Gilliver (John)" wrote

| You're right - I'd forgotten I have an indication (both graphical and
| audible) (both directions) of net traffic, and that's silent, so it
| isn't mining. I didn't think it was, really. (Though would mining
| involve a lot of net traffic, or just a bit to fetch and a bit to return
| the results?)

It seems to be flexible. I imagine something could download
a chunk to chew on, but micro-mining is being done on
webpages by just running client-side script while you're
visiting. That's even been talked about as a possible income
model for commercial sites. So it must be feasible to do tiny
bits efficiently.

| They're different purposes: Everything works on filenames, Agent Ransack
| on file contents.

AR is both. I got it because it *can* search file content.
But it also does file name search very fast. I use both
functions. But I just don't have as many files as most
people seem to. And I organize them. So I don't need
to search so much for file names.

| I don't think Everything - because of the way it
| works, which I don't understand, but it's something to do with NTFS, I
| think - _does_ hammer the disc. I use it mostly - but not exclusively -
| when I want to ask myself "have I already downloaded a file with x
| [often a serial number] in its name".

I'm surprised it needs to index if it doesn't do
content search. Can it actually be faster to search
its own list than to search the file system?

In message , Mayayana
writes:
[Everything]
I'm surprised it needs to index if it doesn't do
content search. Can it actually be faster to search
its own list than to search the file system?

I think maybe it _does_ search the file system; certainly, when you are
typing in the part filename you are looking for, it seems to amend the
list it's presenting to you with each character you type, even at normal
typing speed.

Go on, give it a try - you can always uninstall it. (And I agree,
Everything is a confusing name!)
--
J. P. Gilliver. UMRA: 1960/1985 MB++G()AL-IS-Ch++(p)Ar@T+H+Sh0!:`)DNAf

"Farc gorillas who live in the plains of the undies ..." - automatic
subtitling seen on BBC one o'clock news, 2016-8-25, by Cynthia Hollingworth.

J. P. Gilliver (John) wrote:
In message , Mayayana
writes:
[Everything]
I'm surprised it needs to index if it doesn't do
content search. Can it actually be faster to search
its own list than to search the file system?

I think maybe it _does_ search the file system; certainly, when you are
typing in the part filename you are looking for, it seems to amend the
list it's presenting to you with each character you type, even at normal
typing speed.

Go on, give it a try - you can always uninstall it. (And I agree,
Everything is a confusing name!)

"Everything.exe" indexes in two stages, if it is starting
from scratch.

It reads the $MFT and parses it. This gives a list
of file names, but not their size or creation date.
This might take two seconds. The voidtools designers
really should have stopped at this point.

It's the second phase which is more expensive. If they
want to add dates and sizes and so on, file details,
that requires "walking the tree".

Finally, at some point, the "list" needs sorting.
Perhaps as a means of reducing search time on actual
searches (binary probing?).

None of these activities should particularly leave
the CPU railed, forever... Sorting a list takes
time. But not infinite time.

*******

Later, if a new file is created, it is added to the
NTFS USN Journal. "Everything.exe" hears of this, the item
is added to the list (insertion sort maybe), the size
and date noted. Adding a single file to the existing
index should not be particularly expensive. Only
the first scan of the day should be expensive (assuming
it regenerates the list for fun, once a day).

*******

How does Everything.exe handle Junction Points ?

Does it use a timer to "shut down" aberrant behavior ?

On newer OSes and the C: drive, a lot more care must
be exercised to avoid the "usual traps". A Junction Point
can cause looping recursion until the attempt hits a "path too long"
error and the software in question moves to the next file
or directory. This could potentially take slightly longer
on file systems where the 64K path length has been
enabled (Win10? Optional?).

Another "trap" for file traversing software, is to step
into a certain crypto directory, attempt to read a
certain file... and discover it's a named pipe and it
blocks on input and the visiting program stops in its
tracks. Everything.exe should not fall for that, because
Everything.exe does not "read" files, it only stats() them.
Programs like hashdeep have to be aware of things like
that, and the hashdeep command line is filled with various
letter options for all the "traps" Windows can throw at it.

*******

Even ProcMon can't help us in cases like this.

If Everything.exe was "scanning" the file system, then
ProcMon logs all the file system calls. That activity is
visible.

But if someone puts a "list" in memory and does QuickSort
on it, that's a CPU/memory bound activity, creating
zero entries in ProcMon. So we can't know what it is doing.
When an activity carries out no system calls, we are left
in the dark. Now, it's time for the logic analyzer and
"logging every address".

And that stopped being particularly feasible, a long time ago.
I used to have a $1500 sampling head for a $50K logic analyzer
to make this possible, and those *were* the good old days.

The interface, if still present on modern processors, is likely
to slow down execution so much, as to "disturb" whatever it is
you're trying to study. (At one time, we simply traced every address
and data pin. And "back-filled" using disassembly and software
trace analysis. Modern processors have a "thin" debug interface,
which, while it works, it would take a lot of cycles to
trace a single instruction. And if you had a 28 core processor,
you'd have 28 times as much of that to do.)

Paul

On Sat, 16 Mar 2019 19:49:00 +0000, "J. P. Gilliver (John)"
wrote:

In message , Mayayana
writes:
[Everything]
I'm surprised it needs to index if it doesn't do
content search. Can it actually be faster to search
its own list than to search the file system?

I think maybe it _does_ search the file system; certainly, when you are
typing in the part filename you are looking for, it seems to amend the
list it's presenting to you with each character you type, even at normal
typing speed.

Go on, give it a try - you can always uninstall it. (And I agree,
Everything is a confusing name!)


Yes. I usually call it "Search Everything."

"J. P. Gilliver (John)" wrote

| They're different purposes: Everything works on filenames, Agent Ransack
| on file contents.

I got curious and checked out their FAQ. They say it
can search content but that the default usage is really
something like Windows Explorer with filtering. It displays
everything on the system, then filters as you type.

It seems to record file changes in the background,
then updates its database every time it's run. It also
says you can adjust settings to make it simpler. If
you only need names and not file sizes, dates, etc then
you can skip that.

The overall design doesn't make much sense to me,
but to be honest, I've mainly avoided it because 1) Agent
Ransack is so good and 2) the name "Everything" is just to
ridiculous and confusing.

In alt.windows7.general, on Sat, 16 Mar 2019 16:07:29 +0000, "J. P.
Gilliver (John)" wrote:


They're different purposes: Everything works on filenames, Agent Ransack
on file contents. I don't think Everything - because of the way it
works, which I don't understand, but it's something to do with NTFS, I
think - _does_ hammer the disc. I use it mostly - but not exclusively -
when I want to ask myself "have I already downloaded a file with x
[often a serial number] in its name".

I have Ransack but don't it because I really don't look for text in a
file. (But it was good to learn what the word ransack meant. I thought
it just meant to tear everyhing up. I didn't know it meant for the
purpose of finding something.

One should read the help file or whatever for Everything. It is capable
of a lot. For example, if there were a file with the same anme as
John's email address above, C:\ jp 55 uk would be a way to search for
it. It's more powerful than that, but I can't recall all that it can
do.

micky wrote:
In alt.windows7.general, on Sat, 16 Mar 2019 16:07:29 +0000, "J. P.
Gilliver (John)" wrote:

They're different purposes: Everything works on filenames, Agent Ransack
on file contents. I don't think Everything - because of the way it
works, which I don't understand, but it's something to do with NTFS, I
think - _does_ hammer the disc. I use it mostly - but not exclusively -
when I want to ask myself "have I already downloaded a file with x
[often a serial number] in its name".

I have Ransack but don't it because I really don't look for text in a
file. (But it was good to learn what the word ransack meant. I thought
it just meant to tear everyhing up. I didn't know it meant for the
purpose of finding something.

One should read the help file or whatever for Everything. It is capable
of a lot. For example, if there were a file with the same anme as
John's email address above, C:\ jp 55 uk would be a way to search for
it. It's more powerful than that, but I can't recall all that it can
do.

Not only does Agent Ransack search the content of files,
but, it will engage all your CPU cores while doing it.
If you do a content search on an SSD, it can keep the
bus interface on the SSD very busy.

Agent Ransack does nothing in advance. You pay the same price
for each search. The only "improvement" from one search to
another, comes from the System Read Cache, which improves
the I/O rates on repeated searches you might carry out.

Everything.exe caches filenames, dates, sizes. In the index
file. It does not index content, or read file content that
I know of.

Windows Search indexes everything, content, filename,
date, size, and keeps it in a gigabyte-sized database.
It only does content indexing on things it understands.
Text files are easy. Many other file types defy
analysis (try and find a text string in a movie file).
File types with "providers", it is the "provider"
that pulls the text string out of the text file.
Windows Search doesn't "inherently" know how to
carry out some sort of forensic exam. It needs
a lot of help from other pieces of software
(i.e. a piece of software "per type").

Paul

J. P. Gilliver (John) wrote:
In message , Mayayana
writes:
"J. P. Gilliver (John)" wrote

I used to open "Everything" and then leave it running. It takes a few
seconds to open, then settles down; I can use it, and it finds things
amazingly fast. It is an excellent utility!

However, of late: some minutes after I've opened it and all has settled
down, I hear my fan spin up (it is normally idling), and I start Task
Manager to see what's using CPU - and I see Everything.exe is using 25%
CPU

I don't know about mining, but it does index. You should
be able to check the former by just blocking it from going
out.

You're right - I'd forgotten I have an indication (both graphical and
audible) (both directions) of net traffic, and that's silent, so it
isn't mining. I didn't think it was, really. (Though would mining
involve a lot of net traffic, or just a bit to fetch and a bit to return
the results?)

I've never tried Everything. I like Agent Ransack. Extremely
fast with no indexing. But if you have "a lot a lot" of stuff and
do a lot of searching, maybe indexing makes sense. For me,
I usually know pretty much where things are. I'm more apt
to do a search like finding which of 30 files in a folder has the
line of text I remember from an article I'm trying to find. Given
that, I think of indexing as wasteful wear and tear on disks.

They're different purposes: Everything works on filenames, Agent Ransack
on file contents. I don't think Everything - because of the way it
works, which I don't understand, but it's something to do with NTFS, I
think - _does_ hammer the disc. I use it mostly - but not exclusively -
when I want to ask myself "have I already downloaded a file with x
[often a serial number] in its name".

You don't need to use Agent Ransack to look for text in a file. It works
great for just finding files by file name or file date, and without needing
indexing. So it seems to me they can serve similar purposes, except that I
don't want indexing, and if that's required for Everything, I myself
wouldn't use it. The only exception to this I can see is if I had a HUGE
database that needed to be searched frequently.

On Sat, 16 Mar 2019 13:54:51 -0600, "Bill in Co"
surly_curmudgeon@earthlink wrote:

J. P. Gilliver (John) wrote:
In message , Mayayana
writes:
"J. P. Gilliver (John)" wrote

I used to open "Everything" and then leave it running. It takes a few
seconds to open, then settles down; I can use it, and it finds things
amazingly fast. It is an excellent utility!

However, of late: some minutes after I've opened it and all has settled
down, I hear my fan spin up (it is normally idling), and I start Task
Manager to see what's using CPU - and I see Everything.exe is using 25%
CPU

I don't know about mining, but it does index. You should
be able to check the former by just blocking it from going
out.

You're right - I'd forgotten I have an indication (both graphical and
audible) (both directions) of net traffic, and that's silent, so it
isn't mining. I didn't think it was, really. (Though would mining
involve a lot of net traffic, or just a bit to fetch and a bit to return
the results?)

I've never tried Everything. I like Agent Ransack. Extremely
fast with no indexing. But if you have "a lot a lot" of stuff and
do a lot of searching, maybe indexing makes sense. For me,
I usually know pretty much where things are. I'm more apt
to do a search like finding which of 30 files in a folder has the
line of text I remember from an article I'm trying to find. Given
that, I think of indexing as wasteful wear and tear on disks.

They're different purposes: Everything works on filenames, Agent Ransack
on file contents. I don't think Everything - because of the way it
works, which I don't understand, but it's something to do with NTFS, I
think - _does_ hammer the disc. I use it mostly - but not exclusively -
when I want to ask myself "have I already downloaded a file with x
[often a serial number] in its name".

You don't need to use Agent Ransack to look for text in a file. It works
great for just finding files by file name or file date, and without needing
indexing. So it seems to me they can serve similar purposes, except that I
don't want indexing, and if that's required for Everything, I myself
wouldn't use it. The only exception to this I can see is if I had a HUGE
database that needed to be searched frequently.



Your choice of course, but let me ask what you have against indexing.
It works extremely well for Everything.

Ken Blake wrote:
On Sat, 16 Mar 2019 13:54:51 -0600, "Bill in Co"
surly_curmudgeon@earthlink wrote:

J. P. Gilliver (John) wrote:
In message , Mayayana
writes:
"J. P. Gilliver (John)" wrote

I used to open "Everything" and then leave it running. It takes a few
seconds to open, then settles down; I can use it, and it finds things
amazingly fast. It is an excellent utility!

However, of late: some minutes after I've opened it and all has
settled down, I hear my fan spin up (it is normally idling), and I
start Task Manager to see what's using CPU - and I see Everything.exe
is using 25% CPU

I don't know about mining, but it does index. You should
be able to check the former by just blocking it from going
out.

You're right - I'd forgotten I have an indication (both graphical and
audible) (both directions) of net traffic, and that's silent, so it
isn't mining. I didn't think it was, really. (Though would mining
involve a lot of net traffic, or just a bit to fetch and a bit to return
the results?)

I've never tried Everything. I like Agent Ransack. Extremely
fast with no indexing. But if you have "a lot a lot" of stuff and
do a lot of searching, maybe indexing makes sense. For me,
I usually know pretty much where things are. I'm more apt
to do a search like finding which of 30 files in a folder has the
line of text I remember from an article I'm trying to find. Given
that, I think of indexing as wasteful wear and tear on disks.

They're different purposes: Everything works on filenames, Agent Ransack
on file contents. I don't think Everything - because of the way it
works, which I don't understand, but it's something to do with NTFS, I
think - _does_ hammer the disc. I use it mostly - but not exclusively -
when I want to ask myself "have I already downloaded a file with x
[often a serial number] in its name".

You don't need to use Agent Ransack to look for text in a file. It works
great for just finding files by file name or file date, and without
needing indexing. So it seems to me they can serve similar purposes,
except that I don't want indexing, and if that's required for
Everything, I myself wouldn't use it. The only exception to this I can
see is if I had a HUGE database that needed to be searched frequently.



Your choice of course, but let me ask what you have against indexing.
It works extremely well for Everything.

For me it's a general issue with indexing. The indexes may or may not be up
to date, for example, after adding or modifying or removing some files, but
without indexing, what you see is what you get, and there is no waiting for
an index to rebuilt each time. The downside is it takes longer to find
things, admitely. But I can wait for that. Even more especially if I use a
program like FileLocator Pro, which lets me exclude extraneous windows
directories in the search.

I also have indexing turned off on my system too. That's one of the first
things I did when I get a new computer, but in this case it may also be due
tothe potential for hogging some system resources, on occasion.

As I like to say, Less is (or can be) More, one of my general philosophies
of life. :-) (Somewhat of a Luddite, I guess. :-) Which is another
reason I'm still on XP).

On Sat, 16 Mar 2019 16:07:29 +0000, "J. P. Gilliver (John)"
wrote:

In message , Mayayana
writes:
"J. P. Gilliver (John)" wrote

|I used to open "Everything" and then leave it running. It takes a few
| seconds to open, then settles down; I can use it, and it finds things
| amazingly fast. It is an excellent utility!
|
| However, of late: some minutes after I've opened it and all has settled
| down, I hear my fan spin up (it is normally idling), and I start Task
| Manager to see what's using CPU - and I see Everything.exe is using 25%
| CPU

I don't know about mining, but it does index. You should
be able to check the former by just blocking it from going
out.

You're right - I'd forgotten I have an indication (both graphical and
audible) (both directions) of net traffic, and that's silent, so it
isn't mining. I didn't think it was, really. (Though would mining
involve a lot of net traffic, or just a bit to fetch and a bit to return
the results?)

I've never tried Everything. I like Agent Ransack. Extremely
fast with no indexing. But if you have "a lot a lot" of stuff and
do a lot of searching, maybe indexing makes sense. For me,
I usually know pretty much where things are. I'm more apt
to do a search like finding which of 30 files in a folder has the
line of text I remember from an article I'm trying to find. Given
that, I think of indexing as wasteful wear and tear on disks.

They're different purposes: Everything works on filenames, Agent Ransack
on file contents.


No. Everything works on file names. Agent Ransack works on file
contents *or* file names. Agent Ransack is obviously better for
contents, since Everything doesn't do that, but Everything is *much*
faster (as I'm sure you know) for file names.

"Ken Blake" wrote

|
No. Everything works on file names. Agent Ransack works on file
contents *or* file names. Agent Ransack is obviously better for
contents, since Everything doesn't do that, but Everything is *much*
faster (as I'm sure you know) for file names.
|

----------------------------------------
https://www.voidtools.com/faq/#does_..._file_contents

Does Everything search file contents?

Yes, "Everything" can search file content with the content: search function.
File content is not indexed, searching content is slow.
----------------------------------------

It seems that the best way to think of Everything is
as a modified Explorer program, while AR is a search
program.