PCbanter

PCbanter (http://www.pcbanter.net/index.php)
-   Windows 7 Forum (http://www.pcbanter.net/forumdisplay.php?f=48)
-   -   What's "Everything" doing? (http://www.pcbanter.net/showthread.php?t=1104740)

J. P. Gilliver (John)[_4_] July 17th 18 11:38 AM

What's "Everything" doing?
 
I just found Everything using most of one of my cores, after leaving the
machine unattended - but on - for some hours. It showed in task manager
as using 2x% of CPU (Task Manager thinks I have four cores), and when I
shut it down (via the tray icon, not from Task Manager), CPUsage went
back to idle.

I've just reopened Everything (once I have opened it, I normally leave
it open, as it's so useful), and (after initial peak) it's sitting there
down at near enough zero CPU usage again.

I've noticed this behaviour more than once recently. Anyone know what
it's doing? FWIW, it's "Version 1.4.1.895 (x86)".

(Hour or two later: it's still down in the 00 usage.)
--
J. P. Gilliver. UMRA: 1960/1985 MB++G()AL-IS-Ch++(p)Ar@T+H+Sh0!:`)DNAf

A man is not contemptible because he thinks science explains everything, and a
man is not contemptible because he doesn't. - Howard Jacobson, in Radio Times
2010/1/23-29.

Paul[_32_] July 17th 18 12:29 PM

What's "Everything" doing?
 
J. P. Gilliver (John) wrote:
I just found Everything using most of one of my cores, after leaving the
machine unattended - but on - for some hours. It showed in task manager
as using 2x% of CPU (Task Manager thinks I have four cores), and when I
shut it down (via the tray icon, not from Task Manager), CPUsage went
back to idle.

I've just reopened Everything (once I have opened it, I normally leave
it open, as it's so useful), and (after initial peak) it's sitting there
down at near enough zero CPU usage again.

I've noticed this behaviour more than once recently. Anyone know what
it's doing? FWIW, it's "Version 1.4.1.895 (x86)".

(Hour or two later: it's still down in the 00 usage.)


Anything that "indexes" automatically, can get
into a loop.

Say, for example, MsMpEng keeps an etw file, keeping
track of what was scanned. It makes an update to the
file.

Everything checks the USN journal, and notes the .etw
file got larger. Everything.exe updates its index file for
C: .

MsMpEng sees Everything.exe writing to the index file.
It scans the index file. It adds information to the
..etw to note that it scanned the index file.

Etcetera.

What this means is, exceptions have to be added somewhere,
to stop such behavior.

The Windows Search indexer, obviously doesn't index the
1GB Windows.edb file that holds the inverted index. Because
it would get into a loop. As a result, you can never search
for "Windows.edb" and expect to find it. Unless the search
is in the mood to "poll" unindexed areas.

You might also set up Everything.exe to not use the USN
journal, and only index the disks once a day or something.
But this isn't ideal either, as your indexes won't be
up-to-date when you need them.

Anyway, that's an illustration. You could use Process Monitor,
the ReadFile and WriteFile events, to note which
programs are playing table tennis with which files. And
concoct a theory covering the behavior.

It could always be something entirely different. Like an
actual program bug of some sort. Maybe it's run out of
some resource. There are all sorts of other possibilities.
But "index looping" is something you have to consider for
stuff like this.

And Windows Search gets into loops too. Not CPU grinding
loops, but you can see "293,756 files indexed", then
"293,757 files indexed", then "293,756 files indexed".
And it would do that all day long.

One file that gets erroneously updated frequently, is
a stupid Logitech driver, that even when your webcam
isn't plugged in, it keeps writing entries in a log.
And of course, this leaves USN journal tracks for
the rest of the machine to marvel at. I'd fix
that... if I had source code.

Paul

J. P. Gilliver (John)[_4_] July 17th 18 01:16 PM

What's "Everything" doing?
 
In message , Paul
writes:
[]
Anything that "indexes" automatically, can get
into a loop.

[]
Or, strictly, if you have _two_ such indexing utilities running (e. g.
Everything and something that's intrinsic to Windows). Understood.

What this means is, exceptions have to be added somewhere,
to stop such behavior.


I _suspect_ that's not possible as I _suspect_ it's triggered by
different things, and I'd be playing whack-a-mole (and building an
unwieldy exceptions table/list/whatever).
[]
You might also set up Everything.exe to not use the USN
journal, and only index the disks once a day or something.
But this isn't ideal either, as your indexes won't be
up-to-date when you need them.


Indeed.

Anyway, that's an illustration. You could use Process Monitor,
the ReadFile and WriteFile events, to note which
programs are playing table tennis with which files. And
concoct a theory covering the behavior.


Life's too short (-:
[]
I'll just keep an eye on it. I wouldn't even worry about it if I wasn't
worried about overheating after my last computer; on this one, the
internal fan seems well-controlled, and my usual indication that
something's using more CPU is when I hear the fan step up. (I don't even
know whether it's CPU-load-triggered or temperature-triggered, though
I'd guess the latter.)

FWIW, Everything has now been running for several hours since I
restarted it, and is still in the 00 range.
--
J. P. Gilliver. UMRA: 1960/1985 MB++G()AL-IS-Ch++(p)Ar@T+H+Sh0!:`)DNAf

"If even one person" arguments allow the perfect to become the enemy of the
good, and thus they tend to cause more harm than good.
- Jimmy Akins quoted by Scott Adams, 2015-5-5

Ken Blake[_5_] July 17th 18 03:47 PM

What's "Everything" doing?
 
On Tue, 17 Jul 2018 11:38:03 +0100, "J. P. Gilliver (John)"
wrote:

I just found Everything using most of one of my cores, after leaving the
machine unattended - but on - for some hours. It showed in task manager
as using 2x% of CPU (Task Manager thinks I have four cores), and when I
shut it down (via the tray icon, not from Task Manager), CPUsage went
back to idle.

I've just reopened Everything (once I have opened it, I normally leave
it open, as it's so useful), and (after initial peak) it's sitting there
down at near enough zero CPU usage again.

I've noticed this behaviour more than once recently. Anyone know what
it's doing? FWIW, it's "Version 1.4.1.895 (x86)".



Indexing?

David E. Ross[_2_] July 17th 18 03:53 PM

What's "Everything" doing?
 
On 7/17/2018 3:38 AM, J. P. Gilliver (John) wrote:
I just found Everything using most of one of my cores, after leaving the
machine unattended - but on - for some hours. It showed in task manager
as using 2x% of CPU (Task Manager thinks I have four cores), and when I
shut it down (via the tray icon, not from Task Manager), CPUsage went
back to idle.

I've just reopened Everything (once I have opened it, I normally leave
it open, as it's so useful), and (after initial peak) it's sitting there
down at near enough zero CPU usage again.

I've noticed this behaviour more than once recently. Anyone know what
it's doing? FWIW, it's "Version 1.4.1.895 (x86)".

(Hour or two later: it's still down in the 00 usage.)


Try the following:

1. Launch Everything.

2. Select [Tools Options] from its menu bar.

3. On the left side of the Everything Options window, select General.

4. On the General pane, uncheck all checkboxes except for "Show Search
Everything folder context menu item" and "EFU file association".

5. Select the OK button.

--
David E. Ross
http://www.rossde.com/

Attorney-General Sessions claims the bible favors imprisoning illegal
aliens. However, God repeatedly commanded us to welcome the stranger in
our land. For example, see the following:
Exodus 22:20 at
http://bible.ort.org/books/pentd2.asp?ACTION=displaypage&BOOK=2&CHAPTER=22#P2 131
Exodus 23:9 at
http://bible.ort.org/books/pentd2.asp?ACTION=displaypage&BOOK=2&CHAPTER=23#P2 151
Deuteronomy 10:19 at
http://bible.ort.org/books/pentd2.asp?ACTION=displaypage&BOOK=5&CHAPTER=10#P5 200

J. P. Gilliver (John)[_4_] July 17th 18 04:44 PM

What's "Everything" doing?
 
In message , David E. Ross
writes:
On 7/17/2018 3:38 AM, J. P. Gilliver (John) wrote:
I just found Everything using most of one of my cores, after leaving the
machine unattended - but on - for some hours. It showed in task manager
as using 2x% of CPU (Task Manager thinks I have four cores), and when I
shut it down (via the tray icon, not from Task Manager), CPUsage went
back to idle.

I've just reopened Everything (once I have opened it, I normally leave
it open, as it's so useful), and (after initial peak) it's sitting there
down at near enough zero CPU usage again.

I've noticed this behaviour more than once recently. Anyone know what
it's doing? FWIW, it's "Version 1.4.1.895 (x86)".

(Hour or two later: it's still down in the 00 usage.)


Try the following:

1. Launch Everything.

2. Select [Tools Options] from its menu bar.

3. On the left side of the Everything Options window, select General.

4. On the General pane, uncheck all checkboxes except for "Show Search
Everything folder context menu item" and "EFU file association".

5. Select the OK button.

That might be a cure, but not an explanation as to why it was - when I
came to the computer after hours away - suddenly busy.

FWIW, on that window I currently have (ticks): 11001110101. And it's
using 00 CPU.

To whoever suggested indexing: possible, but when I start it up from not
running, it seems to do whatever it does - I'm assuming something like
indexing, because its list isn't populated until it's finished - in well
under a minute (190,484 objects, it says at the moment). And I think
it's indexing continuously, because if I create (e. g. download) a new
file, I can see it in the Everything list immediately.
--
J. P. Gilliver. UMRA: 1960/1985 MB++G()AL-IS-Ch++(p)Ar@T+H+Sh0!:`)DNAf

What has happened since 1979, I suspect, is that the spotting of mistakes has
become entirely associated with mean-spiritedness, snobbishness and
judgementalism. But...can be...funny and interesting.
Lynn Truss, RT 2015/2/21-27

VanguardLH[_2_] July 17th 18 07:07 PM

What's "Everything" doing?
 
J. P. Gilliver (John) wrote:

I just found Everything using most of one of my cores, after leaving the
machine unattended - but on - for some hours. It showed in task manager
as using 2x% of CPU (Task Manager thinks I have four cores), and when I
shut it down (via the tray icon, not from Task Manager), CPUsage went
back to idle.

I've just reopened Everything (once I have opened it, I normally leave
it open, as it's so useful), and (after initial peak) it's sitting there
down at near enough zero CPU usage again.

I've noticed this behaviour more than once recently. Anyone know what
it's doing? FWIW, it's "Version 1.4.1.895 (x86)".

(Hour or two later: it's still down in the 00 usage.)


Are you running it as a service? It is a file indexer, just like is the
Windows Search service. That means having to monitor and record changes
made through the file system calls (create, rename, move, delete, etc).
When running as a service, you do not need to leave the Everything
window open. In fact, that's just a waste of GUI resources until you
decide to do a file search because Everything is still running in the
background as a service.

In Everything, go to Tools - Options - General: UI tree node. Is the
option enabled to "Run in background"?

When you returned to your computer and Everything.exe got busy, was the
computer sleeping and you woke it up? Low-priority processes or those
configured to not wake the computer will pend their disk I/O until you
wake up the computer (or a BIOS alarm wakes it or a scheduled event in
Task Scheduler is configured to wake the computer).

Paul[_32_] July 18th 18 12:42 AM

What's "Everything" doing?
 
J. P. Gilliver (John) wrote:
In message , David E. Ross
writes:
On 7/17/2018 3:38 AM, J. P. Gilliver (John) wrote:
I just found Everything using most of one of my cores, after leaving the
machine unattended - but on - for some hours. It showed in task manager
as using 2x% of CPU (Task Manager thinks I have four cores), and when I
shut it down (via the tray icon, not from Task Manager), CPUsage went
back to idle.

I've just reopened Everything (once I have opened it, I normally leave
it open, as it's so useful), and (after initial peak) it's sitting there
down at near enough zero CPU usage again.

I've noticed this behaviour more than once recently. Anyone know what
it's doing? FWIW, it's "Version 1.4.1.895 (x86)".

(Hour or two later: it's still down in the 00 usage.)


Try the following:

1. Launch Everything.

2. Select [Tools Options] from its menu bar.

3. On the left side of the Everything Options window, select General.

4. On the General pane, uncheck all checkboxes except for "Show Search
Everything folder context menu item" and "EFU file association".

5. Select the OK button.

That might be a cure, but not an explanation as to why it was - when I
came to the computer after hours away - suddenly busy.

FWIW, on that window I currently have (ticks): 11001110101. And it's
using 00 CPU.

To whoever suggested indexing: possible, but when I start it up from not
running, it seems to do whatever it does - I'm assuming something like
indexing, because its list isn't populated until it's finished - in well
under a minute (190,484 objects, it says at the moment). And I think
it's indexing continuously, because if I create (e. g. download) a new
file, I can see it in the Everything list immediately.


Everything.exe doesn't do content indexing.

At least the versions I've tried didn't.

Originally, it would read the $MFT for file names.
This approach can be lightning fast, from a theory point
of view.

But this doesn't give size or date information. To gather
those requires consulting directory filenums or the like,
one at a time. This required disk head movement.

If Everything does a "wake-up" scan, then it will take
some time to visit the directories and collect the
information.

In the case of "incremental" file info, the USN journal
logs the fact a new file was created, or an old file
was deleted. This allows Everything.exe to add or
delete entries in its index. Everything.exe would still have
to visit the directory, to get size and date, when
files are added.

All of these procedures should be light-years faster
than the content indexing that Windows Search does.

Even finding Everything in a "loop" doesn't make
a lot of sense. It implies Everything has "tangled"
with MsMpEng Windows Defender. Defender can easily
do Ninja stuff, to rail any ordinary process. In
addition, Windows Defender can "jam" programs on
I/O, and they'll stay jammed unless the program
uses a timer to detect such a failure. Most programs
assume, that one way or another, an I/O call will resolve
in 15 seconds, no matter how bad the outcome ("delayed
write failure"). The additional failure modes that
AV programs cause, are not normally considered by
application programmers.

Paul

J. P. Gilliver (John)[_4_] July 18th 18 01:46 AM

What's "Everything" doing?
 
In message , VanguardLH
writes:
J. P. Gilliver (John) wrote:

I just found Everything using most of one of my cores, after leaving the
machine unattended - but on - for some hours. It showed in task manager
as using 2x% of CPU (Task Manager thinks I have four cores), and when I
shut it down (via the tray icon, not from Task Manager), CPUsage went
back to idle.

I've just reopened Everything (once I have opened it, I normally leave
it open, as it's so useful), and (after initial peak) it's sitting there
down at near enough zero CPU usage again.

I've noticed this behaviour more than once recently. Anyone know what
it's doing? FWIW, it's "Version 1.4.1.895 (x86)".

(Hour or two later: it's still down in the 00 usage.)


Are you running it as a service? It is a file indexer, just like is the
Windows Search service. That means having to monitor and record changes
made through the file system calls (create, rename, move, delete, etc).
When running as a service, you do not need to leave the Everything
window open. In fact, that's just a waste of GUI resources until you
decide to do a file search because Everything is still running in the
background as a service.

In Everything, go to Tools - Options - General: UI tree node. Is the
option enabled to "Run in background"?


Yes.

When you returned to your computer and Everything.exe got busy, was the
computer sleeping and you woke it up? Low-priority processes or those
configured to not wake the computer will pend their disk I/O until you
wake up the computer (or a BIOS alarm wakes it or a scheduled event in
Task Scheduler is configured to wake the computer).


No, I didn't wake it up - I don't sleep it; I noticed as I approached
that the fan was running, and when I looked at Task Manager (which I'd
left open), I could see that one core had been busy for more than
however long the Task Manager graphs run for - certainly since before I
touched it. So Everything didn't _get_ busy, it already _was_ busy. And
when I stopped it (from the tray icon), the busyness went away. (And
didn't come back when I restarted - well, it might have for the first
minute or less, but not after that - still isn't, many hours later.)
--
J. P. Gilliver. UMRA: 1960/1985 MB++G()AL-IS-Ch++(p)Ar@T+H+Sh0!:`)DNAf

Intelligence isn't complete without the full picture and the full picture is
all about doubt. Otherwise, you go the way of George Bush. - baroness Eliza
Manningham-Buller (former head of MI5), Radio Times 3-9 September 2011.

VanguardLH[_2_] July 18th 18 03:17 AM

What's "Everything" doing?
 
J. P. Gilliver (John) wrote:

In message , VanguardLH
writes:
J. P. Gilliver (John) wrote:

I just found Everything using most of one of my cores, after leaving the
machine unattended - but on - for some hours. It showed in task manager
as using 2x% of CPU (Task Manager thinks I have four cores), and when I
shut it down (via the tray icon, not from Task Manager), CPUsage went
back to idle.

I've just reopened Everything (once I have opened it, I normally leave
it open, as it's so useful), and (after initial peak) it's sitting there
down at near enough zero CPU usage again.

I've noticed this behaviour more than once recently. Anyone know what
it's doing? FWIW, it's "Version 1.4.1.895 (x86)".

(Hour or two later: it's still down in the 00 usage.)


Are you running it as a service? It is a file indexer, just like is the
Windows Search service. That means having to monitor and record changes
made through the file system calls (create, rename, move, delete, etc).
When running as a service, you do not need to leave the Everything
window open. In fact, that's just a waste of GUI resources until you
decide to do a file search because Everything is still running in the
background as a service.

In Everything, go to Tools - Options - General: UI tree node. Is the
option enabled to "Run in background"?


Yes.

When you returned to your computer and Everything.exe got busy, was the
computer sleeping and you woke it up? Low-priority processes or those
configured to not wake the computer will pend their disk I/O until you
wake up the computer (or a BIOS alarm wakes it or a scheduled event in
Task Scheduler is configured to wake the computer).


No, I didn't wake it up - I don't sleep it; I noticed as I approached
that the fan was running, and when I looked at Task Manager (which I'd
left open), I could see that one core had been busy for more than
however long the Task Manager graphs run for - certainly since before I
touched it. So Everything didn't _get_ busy, it already _was_ busy. And
when I stopped it (from the tray icon), the busyness went away. (And
didn't come back when I restarted - well, it might have for the first
minute or less, but not after that - still isn't, many hours later.)


I've not used the options but there are command-line switches you can
use with everything.exe to produce debug output:

-debug
-debug-log

I think Paul mentioned monitoring file I/O to see what might be altering
the files when Everything gets busy. Updates (writes which result in
changes to file modification datestamps), deletes, and renaming folders,
especially those with lots of files, is expensive to any filename
indexer.

We don't know what processes are running on your computer. As an
example, anything using SQL or any database may occasionally do a purge
and compaction or writing or deleting tons of session data all of which
makes Everything thrash trying to keep up. You should review all
processes that are running to see just what programs are running on your
computer, and then determine if some of them will perform some disk I/O
to maintain themselves. For example, did you disable Windows Indexing
to eliminate that one from running? It will perform a low-priority
background index that Everything might pick up. An anti-virus that
decides to perform a scan, whether scheduled or some other trigger,
could end up updating lots of files. For example, awhile back, many AVs
would create a hash of a file they just scanned to save in the ADS
(alternate data stream) of a file (only doable in NTFS). Later when
another scan was performed, it would rehash the file and compare to the
hash saved the ADS of the file. If the hash matched, the AV would skip
that unchanged file rather than compare it against its signature
database. The skipped files wouldn't get changed but newly scanned
files (since the prior scan) would result in an updated ADS for each
file which Everything might see as a file write operation.

If you run multiple security programs (e.g., two anti-virus or an
anti-virus and an anti-malware) with their on-access (real-time)
scanners active, they can compete with each other. One sees a file got
touched (by another AV), so it scans the file. The other AV sees a file
change so it scans the file. And the vicious cycle repeats. I had that
happen once where within one minute the two AVs had performed over 4000
writes on the same file, and were doing so on multiple files. Tis one
of the reasons that secondary AVs should be passive only: their
on-access scanner is disabled and they are used only for manually
instigated scans.

I've yet to see high CPU usage by Everything except when I use it to do
a search, more so if I use its "content:" operator to dig inside of
files (which will be slow having to open all matching files to do a
substring search). Haven't heard many other Everything users noticing a
high CPU usage by that program. That's why I mentioned you should
probably analyze what processes you leave running on your computer to
see if whatever they do will cause Everything to perform more indexing.
Paul's suggestion to monitor file I/O would point at the culprit doing
tons of file changes (write, read, delete, rename, etc). Just be
careful when you enable the logging. As the log gets bigger, the OS
will slow down. After a couple hours, I was wondering why my computer
had slowed to a crawl, remembered that I had enabled logging in Process
Monitor, stopped the logging, and voila the computer was snappy again.
If you aren't using your computer but Everything is thrashing to perform
lots of indexing then something else is using your computer.

In Everything.ini (there will be a copy in the program installation and
under the %appdata%\Everything folders), you can change max_threads. I
think 0 (zero) means to open as many threads as there are processors
(which would be cores across all CPUs). Setting it less, like 1 less
than the number of cores, would make Everything use less CPU but make
its indexing take longer.

Paul[_32_] July 18th 18 04:50 AM

What's "Everything" doing?
 
VanguardLH wrote:

In Everything.ini (there will be a copy in the program installation and
under the %appdata%\Everything folders), you can change max_threads. I
think 0 (zero) means to open as many threads as there are processors
(which would be cores across all CPUs). Setting it less, like 1 less
than the number of cores, would make Everything use less CPU but make
its indexing take longer.


If your drives are HDD, then you don't need more threads
than spindles.

Blinding indexing partitions in parallel is counterproductive.
You can work in parallel, one thread per spindle, to good effect.
If a HDD has four partitions, inspecting them sequentially makes
sense.

If your computer is filled with SSDs, you can crank that as
high as it will go. With no head assembly to throw around,
"it's all silicon" :-)

You could use Process Monitor, if curious about what the
actual strategy is.

There was one other software product, that got this kinda
wrong at first (it allows the user to set the number of
threads, but also has the option of "using its own discretion").
Hashdeep would use multiple cores, and throw the heads all
over the place while calculating MD5 for all the files
on a disk. Later versions serialized HDD access a bit better,
so the disk didn't thrash disk-wide from the pattern used.

On an SSD, hashdeep can go nuts if it wants (as the hardware
isn't a limitation). (On Win10, you have to disable
Windows Defender if you want some speed out of the thing.
It's yet another application slowed down by WD scanning
in parallel.)

Paul

J. P. Gilliver (John)[_4_] July 18th 18 02:27 PM

What's "Everything" doing?
 
In message , VanguardLH
writes:
[]
We don't know what processes are running on your computer. As an

[many excellent possibilities]

When I left it, and came back to it, nothing more would have been
running than when I actually am using it as I am now - if anything,
slightly less, as I sometimes close some things when I go to bed, though
I don't think I did that time.
[]
I think I'll have to give it up as "one of those things". Though I
_think_ I have seen it do it before (use most of one "core", which went
down again if I closed - and even restarted - Everything).
--
J. P. Gilliver. UMRA: 1960/1985 MB++G()AL-IS-Ch++(p)Ar@T+H+Sh0!:`)DNAf

At the age of 7, Julia Elizabeth Wells could sing notes only dogs could hear.


All times are GMT +1. The time now is 10:37 PM.

Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2004 - 2006 PCbanter
Comments are property of their posters