Home / server / Questions / 914588
In Process
Lieven Keersmaekers
Asked: 2018-06-01 02:19:51 +0800 CST

Interpreting WPR traces in trying to better network throughput

  • 772

In trying to analyze a 3thparty application's throughput we have collected a WPR trace on two systems. One has Symantecs Intrusion Detection System enabled, the other one has not.

Following observations for the system with Intrusion Detection System enabled

  • Many NDIS.SYS - ndisInterruptDpc function calls that take > 100µs.
    Over a period of 10s: 1.966 fragments out of 17.273 took longer than 100µs.
    Microsoft's recommendation for DPC is to not run longer than 100 microseconds.

  • The Microsoft-Windows-TCPIP provider shows everything is using TcpipSendSlowPath.
    I tried reading up on slow/fast paths but not really any idea what to make of this. Everything I can find about it is about routers or dedicated hardware and I doubt ETW can get any data from them (unless it's passed back in the request/response somewhere?).

  • The Stacksview shows the packets going through IDSvia64.sys wich is Symantecs driver for packet inspection. There's a lot more allocating and freeing of Pool memory because of this.

  • psping tests for bandwith are a factor 10 slower.

With that being said: the perceived performance of the 3thparty application on both servers is comparable. The only tangible difference to date are the psping tests being much worse on the system with IDS. A conondrum to me... .

Questions

  • How to find the culprit for taking to much time on DPC's? I suspect IDS is involved but I can not link the handling of DPC's to IDS (or anything else).
  • I would love to know what the Slow path actually means, what performance impact it might have and how to resolve it if possible/needed.

Edit

following the link provided by @Brian, this is how our DPC duration looks like on the system with IDS enabled (green) and on a system without IDS (blue)

  • for a duration of 10s
  • performing a task known to take some time

I'm pretty convinced that IDS is responsible for the longer duration of DPC's. If anyone can give some pointers on the second part of the question for the ammounts of TcpipSendSlowPath, there might be something worth investigating there too.

DPC Duration

A note on the fact that the amount of DPC's on the system with IDS is much larger.

  • The easiest explanation for this is that the system was used at the time by others (it's hard to find a slot with minimum activity)
  • I can imagine the settings of the NIC having a role in this but I wouldn't know where to look for that (depending on buffer size, throughput, ... more ISR's and DPC's getting triggered)
  • others (?) ...
networking

0 Answers