How can I find out what is causing so called "hardware interrupts" on my Windows 7 Professional system?

How can I find out what is causing what TaskInfo by Iarsn System calls
"HW Ints/s", which is hardware interrupts per second?

I typically see a rate of about 11000 HW Ints/s and about 1/2 as many
Thread Sw/s, even when the page fault rate is under 200/second.

I used LatencyMon Home Edition version 6.50 from Respondence Software.
After running for about 30 seconds it shows
DPC count of 106128 for LatencyMon itself
ISR count of 21226 for hal.dll Hardware Abstraction Layer DLL,
DPC count of 4270 for ntoskrnl.exe NT Kernel & System,
DPC count of 3871 for nvlddmkm.sys NVIDIA Windows Kernal Mode Driver,
Version 354.13.
Everything else totals under 6000.

Since TaskInfo would have say about 300000 interrupts in 30 seconds,
not counting the 100000 that LatencyMon made by itself,
I assume that many points are point missed. Perhaps they are the
important data points, perhaps they are randomly distributed.

**Is there a better way to determine the cause of hardware interrupts?


Notes:
I am running Windows 7 Professional with Service Pack 1 and all
Microsoft Updates.

The hardware is a Dell Precision Tower 5810
With Xeon E5-2690 v3 @2.60GHz (12 cores, 24 threads)
64GB memory
System on Samsung 850 PRO 2TB SSD. (This is also
where the work files and any Volume Shadowing stuff
is located.)
Data is on a second Samsung 850 PRO 2TB SSD.
Network is FiOS 150mb/second up, 150mb/second down.
Network benchmarks run by Carbonite support staff see
more than 150mb/s down, 30mb/second down
(30mb/second is more than 3MB/second)
(Multistream backups can actually maintain at
least 120mb/second for days on end in my environment,
so the rest of the load on my network shouldn't affect
Carbonite performance when the household is just watching
TV for a few hours a day and backups are actually transmitting
less than 1 hour a day.)

I have 3 backup programs running at the same time:
Carbonite, CrashPlan, and SpiderOakONE.
I also run all three of these on about 6 other machines,
all of which are slower single thread execution and have
and each of which can only run 2, 4, or 8 threads.

The issue is that Carbonite backup, which is the only thing
besides stuff outside of my control that is running on the system,
after 3 days at about 660KB/second being backed up by Carbonite,
the rate dropped to 34KB/second.

Carbonite seems to be running on only one CPU in one processor
thread. (Therefore the 4%CPU load that shows for it means that
that CPU is maxed out. Note, however, that that same CPU was
originally able to handle 660KB/second.)

I think that I changed a setting or some Microsoft Update caused
a change in performance, but I am not sure. While I know the paging
and I/O rates have remained about the same after the slowdown
compared to the slowdown, I don't know what the old hardware
interrupt rates and task switch rates were.


I realize that the new Xeon may lose more performance due
to task switching compared to my old Core i7 and Pentium systems,
Carbonite did run 20 times faster on this hardware for 4 days or so.

Carbonite wasn't much help, although they did spend time with me.

Mark F wrote:

How can I find out what is causing what TaskInfo by Iarsn System calls
"HW Ints/s", which is hardware interrupts per second?

https://en.wikipedia.org/wiki/Interrupt

Hardware interrupts are used by devices to communicate that they require
attention from the operating system.

This is normal, and has been so for the last, oh, 60 years or so.

--
It must often be so, when things are in danger:
some one has to give them up, lose them, so that others may keep them.

On Sun, 21 Aug 2016 18:24:31 -0000 (UTC), "Auric__"
wrote:

Mark F wrote:

How can I find out what is causing what TaskInfo by Iarsn System calls
"HW Ints/s", which is hardware interrupts per second?

https://en.wikipedia.org/wiki/Interrupt

Hardware interrupts are used by devices to communicate that they require
attention from the operating system.

This is normal, and has been so for the last, oh, 60 years or so.
I know it is normal. Also, as I said originally, I know that the
various paging and actual disk I/O, task switches, and a few other
things are happening at about the same rate as they were before the
problem.

What I am trying to do is identify all the I/O that is happening that
isn't directly related to Carbonite so that I can stop as much as I
can so that I might be able to undo whatever broke things on this
system, or, alternatively, convince the Carbonite people that the
issue is either with their software or use of this particular model of
CPU after some patch was made, or bug in Windows due to a needed patch
or whatever else might be leading to a 20:1 change for the worse in
performance with the same unbroken hardware or 40:1 change between
different machines with unbroken hardware, most of which have slower
disks and CPUs.

I tried stopping things that didn't seem to be needed, but there are
more than 100 programs and services, a pile of which are needed, so
doing a one at a time or Latin square turnoff of things is a big pain
and I'd like an automatic tool for finding the causes anyway.

(By the way: why can't Google, Intuit, Windows, Adobe, and about
10 other things use periodic batch jobs for updates
or just run when the relevant program is running in the first place.
Oh: I don't need faster loading of exe files on SSDs anyhow, but that
is just another thing I would have to turn off every time Windows
decided that saving .01 seconds each of the 1000 times a day I launch
programs, saving me 10 seconds at best/day at best.)

Mark F wrote:
On Sun, 21 Aug 2016 18:24:31 -0000 (UTC), "Auric__"
wrote:

Mark F wrote:

How can I find out what is causing what TaskInfo by Iarsn System calls
"HW Ints/s", which is hardware interrupts per second?
https://en.wikipedia.org/wiki/Interrupt

Hardware interrupts are used by devices to communicate that they require
attention from the operating system.

This is normal, and has been so for the last, oh, 60 years or so.
I know it is normal. Also, as I said originally, I know that the
various paging and actual disk I/O, task switches, and a few other
things are happening at about the same rate as they were before the
problem.

What I am trying to do is identify all the I/O that is happening that
isn't directly related to Carbonite so that I can stop as much as I
can so that I might be able to undo whatever broke things on this
system, or, alternatively, convince the Carbonite people that the
issue is either with their software or use of this particular model of
CPU after some patch was made, or bug in Windows due to a needed patch
or whatever else might be leading to a 20:1 change for the worse in
performance with the same unbroken hardware or 40:1 change between
different machines with unbroken hardware, most of which have slower
disks and CPUs.

I tried stopping things that didn't seem to be needed, but there are
more than 100 programs and services, a pile of which are needed, so
doing a one at a time or Latin square turnoff of things is a big pain
and I'd like an automatic tool for finding the causes anyway.

(By the way: why can't Google, Intuit, Windows, Adobe, and about
10 other things use periodic batch jobs for updates
or just run when the relevant program is running in the first place.
Oh: I don't need faster loading of exe files on SSDs anyhow, but that
is just another thing I would have to turn off every time Windows
decided that saving .01 seconds each of the 1000 times a day I launch
programs, saving me 10 seconds at best/day at best.)

Magic Andre uses "xperf" here to record an ETW event trace.
The output display charges events versus drivers.

http://www.msfn.org/board/topic/1402...dpc-interrupt/

So the ndis.sys seems to have the highest DPC count in that example.
NDIS is for network cards.

(A hardware interrupt, triggers an interrupt handler. The handler
clears the interrupt signal and does a bare minimum of work. A DPC
is scheduled, which runs at user level, to finish the handling of the
interrupt. Thus, there is a relationship between DPCs and interrupts.
The design is intended to minimize the time spent at interrupt level,
keeping the system responsive.)

The MSFN article above has enough links, to get you the Windows 8
Performance Toolkit. The last version of xperf is in there.
I think the Windows 10 edition only has WPA and not the
other tools.

In this message, I played with the kit, but for a slightly
different purpose than you'll be using it for. The kit
insists on .NET 4.0, but that's probably just to make
the installer work :-(

http://al.howardknight.net/msgid.cgi...nt-email.me%3E

Paul