Generating IIS hang dumps when a certain event is triggered in the event viewer

I was working with one of my customer where he had a problem with his website running on a Windows 2003 Server 64 bit and worker processes are running in the 64-bit mode. He was getting "A process serving application pool 'DefaultAppPool' failed to respond to a ping" event in his event viewer. This is a typical hang scenario where the worker process fails to respond to the ping from the W3SVC.

You can use the DebugDiag tool to troubleshoot problems like this and it is well explained in this knowledge base article - http://support.microsoft.com/kb/919792. And, there is version of DebugDiag tool which is available to take hang dumps of the all IIS processes. If it was a 32-bit version, I would've advised the customer to use DebugDiag 1.1 and configure a hang rule for all IIS related processes.

So, here in the 64 bit OS, we are left with only option of using the ADPlus.vbs which comes with the Debugging Tools for Windows.

For my customer, the problem was really sporadic and one can't just monitor the server for event to occur and use ADPlus.vbs to generate hang dump. I was thinking how to help him. There came the savior - EventTriggers. This nice utility available with Windows 2003 Server allows the user to create a trigger for a specific event ID which would fire up another script (.bat or .cmd or anything you would like to execute). Below is the detailed steps to create the same:

1. Download and install the Debugging Tools for Windows 64-bit Version from http://www.microsoft.com/whdc/devtools/debugging/install64bit.mspx
   • The default installation location is C:\Program Files\Debugging Tools for Windows
   • If you are changing the installation location, please follow the step (3) to change the dumpiis.cmd appropriately
2. Please download and extract the AutoDumpGeneratex64.zip - link can be found in the bottom of this post to C:\AutoDumpGeneratex64
   • If you are unzipping it to any other location, open the install.bat in a notepad and edit the last parameter to have the new location
3. Open the dumpiis.cmd in a notepad and change the AdPlus.vbs location to contain the installation location of the Debugging tools for Windows
4. You need to be an Administrator on the machine to perform the below steps:
   • Open a command prompt – change directory (cd) to C:\AutoDumpGeneratex64 folder or the one where you have extracted the files
   • Type “install.cmd”
   • This will prompt you to enter your password for the logon user – enter the password and press enter
   • You should see the below output on the command prompt
     • “SUCCESS: The event trigger "AutoGeneratex64_IIS_dumps" has been created.”
5. Now, if any event for the event ID : 1010 occurs, this will auto trigger the dumpiis.cmd which would take manual hang dumps of all the IIS process running.

I've given the commands written inside the .cmd files here itself. Feel free to use them if you would like to.

install.cmd

eventtriggers /Create /TR "AutoGeneratex64_IIS_dumps" /EID 1010 /L SYSTEM /T ERROR /TK C:\AutoDumpGeneratex64\dumpiis.cmd

dumpiis.cmd

cscript.exe "C:\Program Files\Debugging Tools for Windows\ADPlus.vbs" -hang -iis -o C:\AutoDumpGeneratex64 -quiet