This blog has moved

This blog is now located at http://kiwidudeuk.blogspot.com/.
You will be automatically redirected in 30 seconds, or you may click here.

For feed subscribers, please update your feed subscriptions to
http://kiwidudeuk.blogspot.com/feeds/posts/default.

NCover v2.0.1 (with NCoverExplorer v2.0)

Woah, what's this? You wait six months for the next NCoverExplorer bus to arrive and then two come along at once? Even better, there is a bright and shiny new NCover 2.0.1 to go with it...

Peter Waldschmidt nicely summarises in the new NCover blog his thoughts and rationale behind the decision to transition to a commercialised product. We've been talking about the combined possibilities for a little while now until the timing came right for Peter to make it all happen and start the Gnoso team.

For my part, I'm excited at what this means for the future of both products. Peter and his team have a long list of ideas and enhancements to really drive NCover forward and enhance it's existing reputation as the leading tool for .NET code coverage analysis. This opens up even more possibilities for new features for the NCoverExplorer tools to further improve their ability to deliver quality information to developers. Most importantly, the new business model Peter has put in place offers the necessary resources to make it happen.

You can already see the benefits of this in NCover 2.0.1 which we think is a giant step forward from the betas in 1.x. The highlights listed in "What's New" include a bunch of cool new features like interactive HTML reports, branch coverage and regular-expression based exclusions. I've provided my fair share of support in the NCover forums over time and the important "pain points" from 1.x or missing features like x64 support, IIS and service profiling have been focused on for NCover 2.0. Tremendous effort has gone into an ever growing automated suite of integration and regression tests across a range of OS/.NET versions to keep pushing the bar higher as the product moves forward. With dedicated support now available to address any issues as they arise I'm optimistic the product will continue to evolve very quickly.

So what does this all mean for NCoverExplorer? Well, NCoverExplorer 1.4 will continue to be freely available but will not be further enhanced. Instead my time and effort in conjunction with the Gnoso team has moved onto NCoverExplorer 2.0 which is now bundled with and more tightly integrated into NCover 2.0.1. The last few months have been spent pulling this together and it's a great feeling to get into the product some long awaited and often requested features in this version alone, such as...

• support for all the new NCover 2.0 features (of course!)
  
• multiple "project" support for settings per application you profile
  
• auto-selecting your project settings when coverage xml files are loaded
  
• more reports and enrichment of existing ones
• more coverage exclusion types
• a new and improved NCover dialog
• a graphical source code navigation bar
• an updated and more customisable GUI
• enhanced NAnt/MSBuild tasks
• numerous minor tweaks to enhance ease of use

I will blog about many of these features in more detail over the coming few weeks...

Not to be forgotten is some all important documentation available at long last. Configuring the NCover / NCoverExplorer tools into your build process has always involved a potential number of moving parts. The tool suite includes two console apps and a GUI application, two NAnt tasks, two MSBuild tasks, multiple stylesheets - we will continue to enhance the docs and examples to make this all easier for you.

So, where to from here? Well, by all means please download the trial version of NCover 2.0.1 and send us your feedback. Obviously the new support forums are the ideal place for your questions, feature requests and anything you want to see improved further.

Enjoy! Oh yeah, and don't forget to encourage your coworkers / boss / accountant / significant other / postman etc. to purchase NCover 2.0 licenses if you don't already qualify for a free one... you know you want to...

Filed in: ncoverexplorer ncover

NCoverExplorer... v1.4.0

I've had this "quietly available" on the download page for a few weeks to gain feedback on any issues and with no known issues outstanding it seems appropriate to more publicly announce it.

A big change in NCoverExplorer 1.4.0 is that it is no longer published under the GPL license. I made this change for a number of reasons; amongst which was the concern that a number of companies have of deploying GPL based tools into their development process. This roadblock has been removed and should no longer prevent your companies from using this toolset.

My original choice of GPL was based on using the ICSharpCode 1.x texteditor control. I have now dropped that component in favour of using the superior (but commercial) Actipro SyntaxEditor control instead. The improvements are not only in stability and features but also visual as you can see below in the code highlighting screenshot. A splitter window is available as also shown.


So why should you upgrade to NCoverExplorer 1.4?

This release is less about adding new features (although there are some to be found) and more about stabilisation for the future. The core model code has been rewritten and better tested. There have been countless code coverage corner cases (say that five times in a row!) arise that should be better catered for. Better performance for .NET 2.0 users, lower resource usage, better error handling and some minor gui quirks addressed.

Some of the "internal" changes have spilled over externally. For instance the configuration file format used by NCoverExplorer.Console with /c has changed to support new features, although I have retained legacy compatibility for those who choose not to upgrade their build scripts immediately.

What about the new features? Well, the ones I did squeeze in were...

• Support for the //pm (profiling a process by name) argument in NCover 1.5.8;
• Copy command in the source code text area via right-click;
  
• Print preview command in the File menu;
• /fc argument for NCoverExplorer.Console to fail a build if the combined total coverage of all assemblies does not meet the minimum specified. This was the original behaviour of the /f command before it was changed to fail if any individual assembly does not meet the minimum;
• Coverage exclusions now support the '?' wildcard and multiple '*' wildcards within the expression.
• Report xml contains a "totalSeqPoints" attribute as requested for developers writing alternate xsl reports that need to calculate the excluded code at each level;
  
• Update the NAnt/MSBuild task dlls to support the latest changes;
  
• Redesigned the options dialog as shown below;
  

Downloads
Release Notes

Filed in: ncoverexplorer ncover

CruiseControl.Net... Release 1.3.0 With Integration Queues

Some of you may remember this post from last October about a patch I made for serialising builds in CruiseControl.Net. Owen Rogers was kind enough to give me the "keys to the kingdom" to get this integrated into the main CC.Net branch and as of today it is now officially released in CC.Net 1.3.0.

Having had several pair sessions watching Owen "fly" with Resharper was a great experience - many thanks Owen for the effort in getting the final release out as it can't be easy with new baby and work pressures!

As I said from the very beginning this particular iteration doesn't solve everyone's problems of complex build dependencies, nor was any attempt made to do so. However applying the 80/20 rule it appeared many people were like us with the simpler "this project should never build at the same time as this project" requirement. This implementation should do that just fine with a few bells and whistles such as viewing queue contents, controlling the build order and cancelling pending builds...

We have been using the nightly builds of 1.3 for quite a number of months now and it has proven solid from our experiences with it. There are of course further enhancements that can be made, but in discussion with Owen we froze it as is for people to get started with it and feedback their own thoughts. For instance one addition I would like to see is to identify in the CCTray queue which items are continuous triggers (checking for modifications) versus which are actual builds that will take place. Perhaps a simple "F" stamped into the icon corner for forced builds with additional information being passed in the xml request from the server about the integration request.

Documentation for how to use the integration queue feature is here.

Filed in: CruiseControl.Net

NCover Problems / Fixes - Part 2

In Part 1 of my "everything you wanted to know about NCover but were afraid to ask" series I covered the basics of how NCover works and profiler registration issues. Let's hit another of the most common problems:

Empty Coverage.xml File

This is quite an interesting one, as there are a number of possible causes. You perform your coverage run and don't see any errors in the console output. Yet when you open your coverage.xml file you see something like this from NCover 1.5.7.

<?xml version="1.0" encoding="utf-8"?>
<?xml-stylesheet href="coverage.xsl" type="text/xsl"?>
<!-- saved from url=(0022)http://www.ncover.org/ -->
<coverage profilerVersion="1.5.7 Beta" driverVersion="1.5.7.0" 
startTime="2007-04-19T18:38:47+01:00" measureTime="2007-04-19T18:38:47.921875+01:00" />

If using an earlier version of NCover the xml will be even less as there will be no attributes on the <coverage> node. So what has gone wrong?

Fundamentally it comes down to one thing - there were no debug symbol files found for assemblies that NCover thought it should be profiling. There are multiple reasons as to why this might be.

Debug Symbol Files

Ok, so not everyone knows what a .pdb file is and what purpose it serves. A program database (PDB) file holds debugging and project state information (Source: MSDN), such as line numbers, source code file information and sequence points. This is what NCover needs to produce in the coverage.xml file the line, column and filename information for each sequence point profiled in each method.

If you are compiling a windows or console application with a "Debug" configuration in VS.Net then by default a .pdb file will be produced for each of your assemblies. You can control this in the project properties dialog. Note that by default for "Release" builds no pdb file is produced - another common thing people forget.

For those of not using VS.Net, you can also produce these files using the command line compilers using the /debug option.

When NCover runs, it looks in the current directory of the loaded assembly for a matching debug file. If it doesn't find the .pdb for an assembly, you will see a "Failed to load debug symbols" message in the coverage.log file (more on the log in a future post). NCover then adds that assembly to it's internal list of ones to "skip" and will not bother attempting to profile it.

You can use this to your advantage. The less assemblies being profiled, the smaller the resulting coverage.xml file and the faster it will run. So rather than using the //a argument for NCover to specify assemblies (see below) you could simply delete the .pdb files from your testing bin folder prior to the coverage run for assemblies you do not want profiled.

Having ensured that you are correctly generating .pdb files in your output folder with the assemblies, what circumstances can lead to NCover saying it cannot find them?

- Your assembly was installed in the Global Assembly Cache (GAC)
- You are shadow copying assemblies such as when running nunit.

Profiling GAC Assemblies

When an assembly is installed into the GAC using gacutil it is physically copied to a physical folder location in:

%windir%\Assemblies\GAC\<simple name>\<version>_<culture or blank>_<public key token>

However what isn't so immediately obvious is that the .pdb file is not installed into that same folder by default. So when NCover runs and your assembly is loaded from the GAC, then NCover will not find the .pdb and will skip profiling for it. For the same reason you will have problems trying to use a debugger with that assembly as well.

So how can you get around this? Obviously uninstalling from the GAC is one option and certainly the easiest. I appreciate this isn't always the most convenient though.

Plan B is to move the mountain to Mohammed - and copy the .pdb files into the physical GAC folder. Unfortunately Microsoft didnt make that very easy for us but in typical fashion a number of people out there found a way. You can figure out the location using the pattern above - using the command line of course. Patrick Cauldwell has a blog entry and script for automating it with Powershell (Monad) here. Also Scott Colestock has a NAnt task for doing this here.

NUnit Shadow Copying

This scenario is one I only figured out today in response to an issue posted on the NCover forums here.

By default NUnit will shadow copy your assemblies to a shadow cache folder for running it's unit tests - no shock/horror hopefully in that statement. What isn't so obvious however is the behaviour if you compile your assemblies and .pdb files on one machine and then test them on another.

It appears that .NET does not copy the .pdb files to the shadow cache when they were compiled on another machine - only the assemblies.

So your unit tests still happily pass - but you get no coverage information. Yikes!

Why does it happily copy them on the same machine but not on a different one? I haven't had time to figure that out yet. My guess is some sort of security/signing type of issue but that is just pure speculation. If you know please tell me - I've put a thread on the nunit forums here.

What is the workaround? Either profile your assemblies on the same machine you compiled them on, or run nunit-console with the /noshadow argument. Works a treat.

Choosing Assemblies To Profile

NCover will by default profile all assemblies loaded into memory for which it finds .pdb files. If that path includes a lot of debug files for other assemblies that you aren't interested in getting coverage information for, then this will both harm your profiling performance and give you massive coverage files.

You may also intentionally want to exclude layers of your application - perhaps the presentation or unit test assemblies are deemed irrelevant to your coverage results. I exclude test assemblies from our projects as I feel they unfairly distort the overall coverage percentage. I appreciate others like to have them so they can spot how much of their unit test code was not run.

I mentioned above that a "quick and dirty" trick is to delete the .pdb files for assemblies you do not want coverage for. A more gentlemanly approach is to use the //a argument with NCover. However there are a couple of tricks to it.

Firstly, note that this is the AssemblyName, not the physical assembly file name! No path information, no .dll suffixes etc. A common mistake is to do something like this:

ncover.console.exe nunit-console.exe foo.tests.dll //a foo.dll

If you do this, then you will get an empty coverage file - as NCover did not find a matching AssemblyName. Almost certainly the correct command line for the above is:

ncover.console.exe nunit-console.exe foo.tests.dll //a foo

If you have multiple assemblies you want to report on, then they are separated by semi-colons.

ncover.console.exe nunit-console.exe foo.tests.dll //a foo;bar

Building this list up in a script dynamically can be a minor hassle - thankfully my NAnt and MSBuild tasks for NCover will take care of this for you. In fact, they allow you to use a physical file matching pattern to specify the assemblies to be profiled - the tasks then internally strip all the extraneous information and pass just what NCover expects.

Filed in: ncover

NCover Problems / Fixes - Part 1

From time to time I try to chip in with support on the NCover forums, not only with NCoverExplorer issues but NCover as well since they are so dependent. There are a bunch of issues which come up repeatedly, particularly for people new to using NCover.

In a perfect world (future release?) NCover itself could offer some more helpful error messages, but where is the fun in that? For now at least it is more a case of deciphering the clues from the console output and coverage.log files.

Hopefully this series of posts may help a few people and reduce the times I end up repeating myself on the forums, so win-win if it works for you. My examples will use NCover 1.5.7. You should also take a look at the NCover FAQ which is included in your installation. I've got a copy here too if that helps...


Profiled process terminated. Profiler connection not established.

Easily the most common problem people stumble on. As shown below your application starts (in this case unit tests), but after a delay of 60 seconds or so you get this message.

>E:\apps\NCover\ncover.console.exe E:\apps\nunit\nunit-console.exe MyTests.dll
NCover.Console v1.5.7 - Code Coverage Analysis for .NET - http://ncover.org
Copyright (c) 2004-2006 Peter Waldschmidt

Command: E:\apps\nunit\nunit-console.exe
Command Args: "MyTests.dll"
Working Directory:
Assemblies:
Coverage Xml: Coverage.Xml
Coverage Log: Coverage.Log

Waiting for profiled application to connect...
NUnit version 2.2.8
Copyright (C) 2002-2003 James W. Newkirk, Michael C. Two, Alexei A. Vorontsov, Charlie Poole.
Copyright (C) 2000-2003 Philip Craig.
All Rights Reserved.

OS Version: Microsoft Windows NT 5.1.2600 Service Pack 2 .NET Version: 2.0.50727.42
.
Tests run: 1, Failures: 0, Not run: 0, Time: 0.016 seconds

Connected
Profiled process terminated. Profiler connection not established.

Most likely this is due to one of two reasons I know of:

(1) You are profiling a windows service or IIS - this feature has been reported as currently broken.

(2) You are profiling an executable - but Coverlib.dll was not COM registered

Unfortunately the windows service/IIS testing fell down the cracks for 1.5.7 for those of you trying to use this feature. You can monitor some of the threads on it here and here. Personally for my applications I remove the need to profile services at all. By making the windows service a super lightweight wrapper and moving all the implementation out into classes that are more easily tested there is no need to unit test what remains.

I appreciate however there are scenarios for people where they do want coverage of services for integration testing etc - all I can say for now is that Peter is on the case. Let's now focus on the more common scenario for people trying to profile executables and offer some background info.

NCover Basics

So what's that registration bit about then - surely NCover is a .NET application? Sure, some of it is - but the really clever bit of NCover is the C++ implementation of ICorProfiler in Coverlib.dll, which is a callback class for use with the .NET Profiler API. The requirements of the API are that the callback class has to be COM registered and it cannot be written in managed code. By using some environment variables set within NCover before launching your process, the .NET profiler is able to be "switched on" to make the API callbacks while your process is running.

This is the heart of NCover - it monitors the loading of AppDomains and modules, intercepts the JIT execution of methods and injects it's own instructions to monitor the execution counts. This data is then pumped back to the "managed code" part of NCover, which munges the data together to produce the coverage.xml output you know and love for NCoverExplorer to manipulate.

So the "profiler connection not established" message indicates that there was some sort of problem with communication between NCover and the .NET profiler. As illustrated with the windows service problem above this message doesn't always mean that it was caused by a registry problem, but many times it is and it's worth double-checking first.

NCover Installation / XCopy Deployment

If you installed NCover using the MSI installer, then it will have automatically COM registered CoverLib.dll for you in "C:\Program Files\NCover\". If however, you are like many people intending to run NCover using an xcopy style deployment mechanism as with other tools such as NUnit, then you must remember about this requirement.

So let's say you do want to relocate NCover to a tools subfolder in a source controlled directory - what's the idiot-proof way to do it?

Firstly, make sure you copy all the NCover files from the installation folder above. Make sure you don't skip the Microsoft.VC80.CRT.manifest, MSVCP80.dll or MSVCR80.dll files. If you do skip them, then an attempt to register Coverlib.dll will fail on machines without these .dlls already in a path somewhere. You may find it works on your development machine with VS.Net installed, but then fails on your build server.

Secondly, make sure you run the NCover uninstall which will clean up the registry as well as the installed files. So now your machine should be back in "pristine" condition.

Options For Registering NCover

So having identified that when running NCover.Console.exe you need to ensure that Coverlib.dll is registered - what are your options?

1. If you stuck with the default NCover installation, have no need for locating it elsewhere and need only one version of NCover for all your applications. Congratulations - you shouldnt have seen the problem above! If you did, you should manually re-register it using regsvr32 or try running the installer again.

2. The NAnt/MSBuild tasks I provide automatically register and unregister Coverlib.dll on the fly under HKCU. So if using those you are already sorted.

3. If using the command line, NCover 1.5.7 now offers a //reg option. This will do the same as the tasks above of registering and unregistering on the fly.

4. You could invoke regsvr32 on it as part of your script. I suggest you unregister it as well afterwards in case you have multiple NCover versions on your machine to prevent any later confusion.

NCover With TestDriven.NET

I got myself caught out with this little gotcha recently and I'm sure I won't be the last!

As you likely know the excellent TestDriven.Net add-in by Jamie Cansdale includes a copy of (currently) NCover 1.3.3 and NCover 1.5.7 bundled with it. What you probably don't know is under which scenarios it runs which version of NCover and where this can go wrong.

The first thing TestDriven.Net does is check the registry to see if you have installed NCover yourself using the MSI installer. If there isn't then it knows to use one of the versions it has installed. If you have .NET 2.0 installed on your machine it will use the 1.5.x version, otherwise it will use NCover 1.3.3.

If however you do have NCover installed from the MSI, then TestDriven.NET assumes that you want to use that version. Jamie's rationale for this is simple - if you ended up with later versions of NCover on your machine than that supplied with TestDriven.NET then it should use that over it's own.

Where this can go wrong as it did for me is if for some reason you have unregistered NCover in that folder. TestDriven.NET makes the reasonable assumption that if you installed it and left it there then you have it registered there too. However as I dabble with testing and supporting multiple NCover versions all over my drives I had manually unregistered it without remembering to uninstall it. That mistake will then lead to error messages from TestDriven.NET about "Unable to create a coverage report". Either fully uninstall that NCover installation or register CoverLib.dll again in that folder to fix this. My thanks to Jamie for his excellent support!

TypeMock And NCover NAnt/MSBuild Tasks

TypeMock is a very powerful mock objects framework which no doubt some of you are using. Assuming you have purchased the commercial version then you will likely want to be running your tests with code coverage using NCover.

The one gotcha with this is that a requirement of the .NET profiling API is that there can only be one profiler callback registered on a machine when it runs. TypeMock takes care of this by managing the registry itself. However if you use my NAnt/MSBuild tasks then by default they want to "undo" the work of TypeMock by registering NCover, causing a problem.

The solution is available from the 1.3.6.11 build of my tasks in NCoverExplorer.Extras, the latest of which is available here. By adding a registerProfiler="false" attribute to your NAnt task or equivalent for MSBuild the default behaviour of the tasks to register in HKCU is turned off and control is able to be maintained by TypeMock.

That's enough for today. Part 2 will cover the mysterious blank coverage.xml file. Feedback/corrections welcomed - if you have an NCover problem your best bet is to head to the NCover forums.

Filed in: ncover

NCoverExplorer... v1.3.6

Some of you may already have noticed the long awaited release of NCover 1.5.7 yesterday - hoorah! That now being out of the way and in conjunction with some easing of my day job pressure, has made it a good time for me to finally rollout an NCoverExplorer update.

Note - if you have downloaded NCoverExplorer or TestDriven.Net over the last week or so you may want to do so again. I have been sneaking in fixes for issues right up until this blog posting. Check the downloads page to make sure you have the very latest build number.

This is predominantly a bug-fix release, addressing a range of issues that have accumulated over the last six months. Of utmost importance was improving the coverage result accuracy from either the merging of multiple coverage.xml files or multiple AppDomains within a single file. Some of you have experienced issues with the "optimisations" of removing noop sequence points that NCover 1.5.4, 1.5.5 and 1.5.7 apply in different ways. This resulted in duplicated nodes in the treeview, or lower than expected coverage with brackets being shown as not hit etc. My thanks to everyone who contributed to the testing of this with their coverage files/repro projects and patience while I got the kinks out.

On the new feature front, the biggest is a "Find Type" dialog (ctrl+F) allowing you to navigate to types by entering their name (supports * for a wildcard).


A change to previous behaviour is to the fail on minimum coverage option for NCoverExplorer.Console. Based on feedback this has now been changed so that any assembly not reaching the threshold will cause the build to fail, rather than total coverage.

All the other tweaks and fixes can be found in the release notes.

I have updated the NCoverExplorer Extras download containing the NAnt and MSBuild tasks. The <ncover> task now uses a reference counting style approach to the registration of coverlib.dll, which should get around issues reported on build servers of simultaneous build issues if you got unlucky with the timings.

The task now includes a registerProfiler optional attribute for those of you using TypeMock - you can set this to "false" to ensure the NCover task does not try to register CoverLib.dll itself.

There is an enhancement to the <nunitproject> task to support the appbase property for people using multiple testing folders (thanks to Garth Williams).

Another addition is a new <ndoc2> NAnt task in there for anyone using NDoc2 Alpha as I blogged about in this post. Documentation is available via the links on this page but anyone familiar with the <ndoc> task should be happy. I've used it for generating CHM documentation without any problems.

For any issues with the release or further enhancement suggestions, please post to the NCoverExplorer forums or send me an e-mail (Help -> Send Feedback).

A quick further mention on NCover 1.5.7. This is a mandatory upgrade for every NCover user as far as I am concerned. Peter Waldschmidt's major rewrite of the profiler core and the addition of an integration test suite sets up a great platform for future releases. I've been fortunate to be able to chip in here and there on this release, although arguably more nuisance than help at times to Peter I'm sure!

I am highly optimistic however that this is the most solid NCover beta to date and was worth the wait to get it right. Top stuff Peter and many thanks for all the effort!

Last but certainly not least, Jamie Cansdale has made available a new build of his excellent TestDriven.Net add-in. Amongst other goodies this now includes the latest build of NCoverExplorer, NCover 1.5.7 and Visual Studio Express SKU support. Right-clicky clicky goodness...

Downloads
Release Notes

Filed in: ncoverexplorer ncover ndoc