Automatic capturing of memory dumps when applications crash in non-persistent environments

Applications crash. Sometimes (especially when it happens more frequently than usual) you want to have these memory dumps. Either to analyze them yourself or to send them to the responsible company.

In my mind, I think of a Citrix PVS environment with many terminal servers, clients, VDAs (call it what you want). But the described way is potentially applicable in other scenarios as well.

Basically, it consists of these four steps:

1. Create a persistent disk
2. Set ACLs
3. Set up ProcDump
4. Automatically clean up the memory dumps

Persistent disk

At least in a Citrix PVS environment, it is common to have a persistent disk (for the overflow on hard disk part). Additionally, it is common to redirect the most important event logs to survive a reboot (if you don’t use central logging anyway).So we can also store the memory dumps there. Example location: D:\CrashDumps.

ACLs for the memory dumps

It is a good idea to secure this directory with ACLs. Memory dumps can contain sensitive information, so you want to make sure that not everyone can read them.

The ACLs must of course allow users to write to the D:\CrashDumps directory.

So a possible configuration could look like this: To make sure that the user has no access rights:

icacls D:\CrashDumps /remove users

Give the users the necessary write permissions:

icacls D:\CrashDumps /grant users:(S,WD)

The flags are described in the help as follows: S — synchronize WD — write data/add file

But of course, there are dozens of other ways to configure this. The important thing is that the memory dumps are not just lying around unprotected.

Set up ProcDump

ProcDump is probably the command-line tool to create memory dumps in different scenarios. To configure ProcDump as a post-mortem debugger, we need to specify the -i parameter. In addition to the path where the memory dump should be stored, so ProcDump -i D:\CrashDumps.
Now you have to decide what exact type of memory dumps you want to create. If disk space is not an issue, -ma option is not a mistake to get everything.
If disk space is quite limited, -mp might also be sufficient for the time being to perform an analysis. You probably end up with something like this: ProcDump -i -ma D:\CrashDumps.

Automatic cleanup

We have now reached the state that as soon as an application crashes a memory dump is created to the persistent disk (secured with ACLs). The problem is that after a certain time the disk would fill up with memory dumps. It makes sense to take care that these are cleaned up regularly. For example, we could run a PowerShell script as a scheduled task every hour. Script content could be as follows:

Get-ChildItem D:\CrashDumps | sort lastwritetime | select -first (((Get-ChildItem D:\CrashDumps).count)-5) | Remove-Item

This one-liner guarantees that 5 memory dumps are kept but if there are more, the older ones are deleted until the most recent 5 remain.

The next time an application crashes you are well prepared and already have the memory dump.