1. .NET Debugging for the Production Environment
Part 3: WinDbg & SOS
OPEN VM BEFORE STARTING THIS SESSION (for demo). Make sure mscordacwks.dll in the Fwk directory and mscordacwks_x86….42.dll in the debuggers directory are mis-named.
Brad Linscott
Premier Field Engineer ing

2. Agenda Windbg intro Debugging .NET applications with Windbg
A common problem when trying to load SOS
In this session, we’ll start with a brief introduction on windbg before we talk about how we can use it to debug .NET applications. And then the focus of today’s talk will be a common problem many folks encounter when they try to start debugging with SOS. If you’re not sure what SOS is, I’ll explain that as well.

3. WinDbg Intro Part of Debugging Tools for Windows package
Free download on Microsoft’s web site
Xcopy-deployable
Automatable (ADPlus), live, post-mortem (production!)
Why use WinDbg/cdb instead of Visual Studio?
- So what is WinDbg? It’s one of the many tools included in the Debugging Tools… The package contains several debuggers, and WinDbg is probably the most popular one. It has a very detailed help file that’s really is helpful. Though it’s not likely to help those who haven’t debugged with these debuggers before, it’s geared towards those who have some experience with them, and simply want to expand their proficiency.
- It’s a free download from Microsoft’s web site (install the 32-bit debuggers), and their designed to be installed on production servers - I’ve worked with countless customers who have them installed on their production servers. They’re xcopy-deployable, so you can download and install them on a machine with internet access and then x-copy them to the target server; no reboot required.
These debuggers are automatable, which makes them a great fit for production. ADPlus is another tool in the Debugging package. It comes in an exe & script format, and it automates the console debugger. These debuggers can be used for live debugs or post-mortem debugging, which comprises the vast majority of our production debugging experience.
Which is a good segue to a popular question – “Why use Windbg instead of VS?” Part of the answer lies in the automation of these production debuggers. Also, VS has a large footprint and isn’t installed on prod servers. The more you use windbg & the others, the more you understand their power & simplicity. They provide a great debugging experience.

4. WinDbg, cont’d Unaware of .NET SOS/psscor2/psscor4
Symbols – needed with managed debugging?
Saving your workspace in Windbg
- Some more notes about Windbg… like any native debugger, it’s completely unaware of the managed world. It can’t peek into the execution of managed code. In order for it to do that, it needs the help of an extension.
A debugging extension is a dll that has a number of “shortcuts”, if you will, to run various debugging commands. There are some extensions whose sole purpose is to provide native debuggers with a way to extract information from managed code. The most popular extension to help with managed debugging is SOS or one of its forms.
(Talk about SOS, how it comes with the framework. )
Psscor2 is downloadable from our web site, as is psscor4.
I’ll take a brief minute to discuss symbols, b/c they’re a must when debugging native code. (Talk about how they’re needed to resolve stacks.) With managed code, symbols aren’t needed to accomplish the vast majority of your debugging goals, but it’s still a good practice to create them for all release builds and store them on a symbol server. If you do ever need them, they must MATCH
So that’s a brief word on symbols, and this is a good time to talk about workspaces. Demo saving symbol path & other settings.

5. .Debugging .NET apps with WinDbg
Command window is “main” window
Load sos/psscor2/psscor4
.load <extension name>
Now we come to the focus of this session – opening a dump and loading SOS.
As stated before, WinDbg can do live debugging or open post-mortem dumps. I’ll walk through a quick sample of opening a dump and loading psscor2.
The Command window is the most commonly used window in windbg. There are several other windows including Locals, Stack, and Source windows. If you have your Source path lined up correctly, you can even view source code in windbg. During live debugs, you can also step through code, set bps, etc.
Discuss .load vs loadby.
When you get the error, go to the next slide.

6. Common problem loading SOS/psscor2/psscor4
“Failed to load data access dll”
Explain that the first arch is the arch of the debugger. 2nd arch is arch of the target exe. Version is version of the fwk.

7. Common problem loading SOS/psscor2/psscor4
What is the ‘dac’?
Common root causes of “failed to load” with sos
Download from symbol server
Not using the right dac – copy from the target machine
Debugger/debuggee architecture mismatch
Talk about mscordacwks
Talk about how this problem almost never occurs for live debugs, b/c the dac is distributed with the Fwk, and so is going to be on the machine of the debugee.
Don’t discuss troubleshooting here – go to next slide for that.

8. Problem loading SOS/psscor2/psscor4, cont’d
Recommended troubleshooting steps:
Use .cordll, as suggested
Ensure the right dac is used
Debugger/debuggee mismatch
Flip to the debugger with the error on it. Try running .cordll. Explain that the .cordll command controls managed code debugging for the CLR. We turn on verbose output, unload it, and try to load back the right one.
If .cordll doesn’t work, then search the Framework path and Debugger location to see if the right dac is present. I keep a stash in the same folder as windbg itself to reduce the number of times I get this error.
The most common root cause is an arch mismatch between the target exe & debugger.

9. Summary Two simple rules when dumping/debugging managed applications:
Dump 32-bit applications with a 32-bit debugger, and open the dump with a 32-bit debugger.
Dump 64-bit applications with a 64-bit debugger, and open the dump with a 64-bit debugger.
Keeping architectures aligned will eliminate this problem 90% of the time
Review the summary.
Next time we’ll investigate something a bit more interesting that may help some of you encountering process instability or a performance problem in production.