Most of us are unprepared for catastrophic failure in our application, that is, the kind of failure that is so pernicious that even our precious log files have nothing useful to share. In fact to get adequate data during these moments we usually need the conditions to be replicated exactly and then hope we are adequately prepared to monitor the details we missed.
For the last couple of years I have been asking our clients to prepare hang dumps for me under a variety of circumstances so that I can ensure I have an accurate portrait of a given situation. In years past we would normally do this using ADPlus, however, there are few clients who are willing to install a subset of the the Windows SDK during or after production outages.
More recently then I have been pushing the more straightforward copy paste installation of ProcDump. It is a popular command-line utility that serves as a general purpose dump utility. There is a whole list of ProcDump parameters to become familiar with but I wanted to share the combinations that I find myself using repeatedly when collecting hang or crash dumps under a variety of circumstances.
Immediately take a full hang dump:
C:\Dev\Procdump>procdump -ma w3wp
Take a full hang dump when the application crashes:
C:\Dev\Procdump>procdump -e -ma w3wp
Write three full hang dumps 10 seconds apart:
C:\Dev\Procdump>procdump -ma -s 10 -n 3 w3wp
Take a full hang dump when the w3wp process exceeds 40%:
C:\Dev\Procdump>procdump -c 40 -s 5 -n 3 w3wp
Take a full hang dump when the overall CPU exceeds 70%:
C:\Dev\Procdump>procdump - ma w3wp -p "\Processor(_Total)\% Processor Time" 70
Write a full hang dump when an exception contains the word "IO" e.g. "IOException":
procdump - ma -e 1 -f IO w3wp
All the previous hang/crash dump strategies are concerned with w3wp, the process associated with the application pool in IIS, however, you can update this for any process name or number you like.