NCover Problems / Fixes - Part 2

In Part 1 of my "everything you wanted to know about NCover but were afraid to ask" series I covered the basics of how NCover works and profiler registration issues. Let's hit another of the most common problems:

Empty Coverage.xml File

This is quite an interesting one, as there are a number of possible causes. You perform your coverage run and don't see any errors in the console output. Yet when you open your coverage.xml file you see something like this from NCover 1.5.7.

<?xml version="1.0" encoding="utf-8"?>
<?xml-stylesheet href="coverage.xsl" type="text/xsl"?>
<!-- saved from url=(0022)http://www.ncover.org/ -->
<coverage profilerVersion="1.5.7 Beta" driverVersion="1.5.7.0" 
startTime="2007-04-19T18:38:47+01:00" measureTime="2007-04-19T18:38:47.921875+01:00" />

If using an earlier version of NCover the xml will be even less as there will be no attributes on the <coverage> node. So what has gone wrong?

Fundamentally it comes down to one thing - there were no debug symbol files found for assemblies that NCover thought it should be profiling. There are multiple reasons as to why this might be.

Debug Symbol Files

Ok, so not everyone knows what a .pdb file is and what purpose it serves. A program database (PDB) file holds debugging and project state information (Source: MSDN), such as line numbers, source code file information and sequence points. This is what NCover needs to produce in the coverage.xml file the line, column and filename information for each sequence point profiled in each method.

If you are compiling a windows or console application with a "Debug" configuration in VS.Net then by default a .pdb file will be produced for each of your assemblies. You can control this in the project properties dialog. Note that by default for "Release" builds no pdb file is produced - another common thing people forget.

For those of not using VS.Net, you can also produce these files using the command line compilers using the /debug option.

When NCover runs, it looks in the current directory of the loaded assembly for a matching debug file. If it doesn't find the .pdb for an assembly, you will see a "Failed to load debug symbols" message in the coverage.log file (more on the log in a future post). NCover then adds that assembly to it's internal list of ones to "skip" and will not bother attempting to profile it.

You can use this to your advantage. The less assemblies being profiled, the smaller the resulting coverage.xml file and the faster it will run. So rather than using the //a argument for NCover to specify assemblies (see below) you could simply delete the .pdb files from your testing bin folder prior to the coverage run for assemblies you do not want profiled.

Having ensured that you are correctly generating .pdb files in your output folder with the assemblies, what circumstances can lead to NCover saying it cannot find them?

- Your assembly was installed in the Global Assembly Cache (GAC)
- You are shadow copying assemblies such as when running nunit.

Profiling GAC Assemblies

When an assembly is installed into the GAC using gacutil it is physically copied to a physical folder location in:

%windir%\Assemblies\GAC\<simple name>\<version>_<culture or blank>_<public key token>

However what isn't so immediately obvious is that the .pdb file is not installed into that same folder by default. So when NCover runs and your assembly is loaded from the GAC, then NCover will not find the .pdb and will skip profiling for it. For the same reason you will have problems trying to use a debugger with that assembly as well.

So how can you get around this? Obviously uninstalling from the GAC is one option and certainly the easiest. I appreciate this isn't always the most convenient though.

Plan B is to move the mountain to Mohammed - and copy the .pdb files into the physical GAC folder. Unfortunately Microsoft didnt make that very easy for us but in typical fashion a number of people out there found a way. You can figure out the location using the pattern above - using the command line of course. Patrick Cauldwell has a blog entry and script for automating it with Powershell (Monad) here. Also Scott Colestock has a NAnt task for doing this here.

NUnit Shadow Copying

This scenario is one I only figured out today in response to an issue posted on the NCover forums here.

By default NUnit will shadow copy your assemblies to a shadow cache folder for running it's unit tests - no shock/horror hopefully in that statement. What isn't so obvious however is the behaviour if you compile your assemblies and .pdb files on one machine and then test them on another.

It appears that .NET does not copy the .pdb files to the shadow cache when they were compiled on another machine - only the assemblies.

So your unit tests still happily pass - but you get no coverage information. Yikes!

Why does it happily copy them on the same machine but not on a different one? I haven't had time to figure that out yet. My guess is some sort of security/signing type of issue but that is just pure speculation. If you know please tell me - I've put a thread on the nunit forums here.

What is the workaround? Either profile your assemblies on the same machine you compiled them on, or run nunit-console with the /noshadow argument. Works a treat.

Choosing Assemblies To Profile

NCover will by default profile all assemblies loaded into memory for which it finds .pdb files. If that path includes a lot of debug files for other assemblies that you aren't interested in getting coverage information for, then this will both harm your profiling performance and give you massive coverage files.

You may also intentionally want to exclude layers of your application - perhaps the presentation or unit test assemblies are deemed irrelevant to your coverage results. I exclude test assemblies from our projects as I feel they unfairly distort the overall coverage percentage. I appreciate others like to have them so they can spot how much of their unit test code was not run.

I mentioned above that a "quick and dirty" trick is to delete the .pdb files for assemblies you do not want coverage for. A more gentlemanly approach is to use the //a argument with NCover. However there are a couple of tricks to it.

Firstly, note that this is the AssemblyName, not the physical assembly file name! No path information, no .dll suffixes etc. A common mistake is to do something like this:

ncover.console.exe nunit-console.exe foo.tests.dll //a foo.dll

If you do this, then you will get an empty coverage file - as NCover did not find a matching AssemblyName. Almost certainly the correct command line for the above is:

ncover.console.exe nunit-console.exe foo.tests.dll //a foo

If you have multiple assemblies you want to report on, then they are separated by semi-colons.

ncover.console.exe nunit-console.exe foo.tests.dll //a foo;bar

Building this list up in a script dynamically can be a minor hassle - thankfully my NAnt and MSBuild tasks for NCover will take care of this for you. In fact, they allow you to use a physical file matching pattern to specify the assemblies to be profiled - the tasks then internally strip all the extraneous information and pass just what NCover expects.

Filed in: ncover