I have been getting really familiar with the Snapshot Debugger in Visual Studio recently and I have to say it is one of the best kept secrets of Visual Studio Enterprise and Azure.
The Snapshot Debugger’s mission is to allow developers to diagnose problems in their production or live environments with minimal impact. It attempts to mirror the same debugging experience that developers are used to. The way Snapshot Debugger works is that users select Snappoints and Logpoints similar to the debugger breakpoints and tracepoints. When the line of code is executed on in your production environment, the process forks and snapshots are created dynamically without stopping the process. Developers can then attach to these snapshots using Visual Studio and see what’s going on with variables.
The Snapshot Debugger currently targets Azure App Services, however, the VS 2019 release will extend support to Azure Virtual Machines, Azure Scale Sets, and Azure Kubernetes Service (AKS). Exciting stuff!
Snapshot Debugging sounds like existing technology, remote debugging and hang dumps come to mind, but I started to imagine it as a hybrid of the two where we take the best features without the apparent weaknesses. I collated this list of questions that helped me decide why I might select this tool first especially in high performance and high availability environments.
My Q&A on Snapshot Debugger
What is a Process snapshot?
You may be more familiar with a fork which is an operation whereby a process creates a copy of itself, this term is normally used in Unix environments but applies broadly across platforms. Process Snapshotting provides the equivalent functionality within a Windows context.
How is Snapshot Debugging different to Remote debugging?
The ability to debug an application running on a different computer has been available within Visual Studio for many years. You would need to download and install the remote tools, however, a key differences is that hitting a break point negatively impacts the running application because everything pauses at your breakpoint.
The Snapshot Debugger was introduced with Visual Studio 2017 (15.5) and it makes a copy (fork) of the remote process and your session analyzes the fork rather than the process directly. Process snapshotting allows you to review a snapshot of your application with negligible impact to the running process.
Whether Remote debugging or Snapshot Debugging you are interacting with the debugging agent (msvsmon.exe) which listens for connections and marshals the breakpoints and snappoints created within your Visual Studio session.
How is a Process Snapshot different to a hang dump?
A hang dump will normally contain the call stacks, registers, threads and data of the entire application, it's a point in time snapshot that includes all the details of the application and also may include details about OS health (Thread pool, memory, CPU, etc.). Taking a hang dump will pause your entire application while it is being created, directly impacting the performance.
Process Snapshotting creates a more streamline type of snapshot, it is not a copy of the full heap of the app, it’s only a copy of the page table which is immediately set to copy-on-write (shadowing) this significantly reduces resource consumption and the impact on the running process is low.
Does the size of the process impact the time taken to get a snapshot?
The overhead of capturing snapshots is fixed and should therefore not affect the throughput of your app regardless of its scale, you may see a slowdown of 10-30ms when creating snapshots. When you set your snappoint that method is deoptimize however the rest of the application stays the same.
Does the size of the process impact the size of the snapshot?
Each snapshot taken with this process has a very low in-memory cost usually a few hundred kilobytes for most apps. This is in contrast to a hang dump where the file created usually matches or exceeds the size of the process' memory footprint.
What is the API used to take a snapshot?
Process Snapshotting - PssCaptureSnapshot
Are there guards against taking a snapshot under load (high memory, CPU, disk, etc.)?
The Snapshot Debugger will only capture one snapshot per snappoint placed in your code (or as configured). You can specify conditional statements to determine when the snapshot should be captured.
A system memory check is also performed prior to creating a snapshot, if it is determined that we have insufficient memory, less than twice the process size, no snapshot will be taken.
When you stop the Snapshot Debugger it will detach freeing all existing snapshots from memory on the server.