Kiwidude's Geek Spot

This blog has moved

2010-04-28T11:40:00.001+01:00

This blog is now located at http://kiwidudeuk.blogspot.com/.
You will be automatically redirected in 30 seconds, or you may click here.

For feed subscribers, please update your feed subscriptions to
http://kiwidudeuk.blogspot.com/feeds/posts/default.

NCover v2.0.1 (with NCoverExplorer v2.0)

2007-09-19T22:19:00.000+01:00

Woah, what's this? You wait six months for the next NCoverExplorer bus to arrive and then two come along at once? Even better, there is a bright and shiny new NCover 2.0.1 to go with it...

Peter Waldschmidt nicely summarises in the new NCover blog his thoughts and rationale behind the decision to transition to a commercialised product. We've been talking about the combined possibilities for a little while now until the timing came right for Peter to make it all happen and start the Gnoso team.

For my part, I'm excited at what this means for the future of both products. Peter and his team have a long list of ideas and enhancements to really drive NCover forward and enhance it's existing reputation as the leading tool for .NET code coverage analysis. This opens up even more possibilities for new features for the NCoverExplorer tools to further improve their ability to deliver quality information to developers. Most importantly, the new business model Peter has put in place offers the necessary resources to make it happen.

You can already see the benefits of this in NCover 2.0.1 which we think is a giant step forward from the betas in 1.x. The highlights listed in "What's New" include a bunch of cool new features like interactive HTML reports, branch coverage and regular-expression based exclusions. I've provided my fair share of support in the NCover forums over time and the important "pain points" from 1.x or missing features like x64 support, IIS and service profiling have been focused on for NCover 2.0. Tremendous effort has gone into an ever growing automated suite of integration and regression tests across a range of OS/.NET versions to keep pushing the bar higher as the product moves forward. With dedicated support now available to address any issues as they arise I'm optimistic the product will continue to evolve very quickly.

So what does this all mean for NCoverExplorer? Well, NCoverExplorer 1.4 will continue to be freely available but will not be further enhanced. Instead my time and effort in conjunction with the Gnoso team has moved onto NCoverExplorer 2.0 which is now bundled with and more tightly integrated into NCover 2.0.1. The last few months have been spent pulling this together and it's a great feeling to get into the product some long awaited and often requested features in this version alone, such as...

support for all the new NCover 2.0 features (of course!)
multiple "project" support for settings per application you profile
auto-selecting your project settings when coverage xml files are loaded
more reports and enrichment of existing ones
more coverage exclusion types
a new and improved NCover dialog
a graphical source code navigation bar
an updated and more customisable GUI
enhanced NAnt/MSBuild tasks
numerous minor tweaks to enhance ease of use

I will blog about many of these features in more detail over the coming few weeks...

Not to be forgotten is some all important documentation available at long last. Configuring the NCover / NCoverExplorer tools into your build process has always involved a potential number of moving parts. The tool suite includes two console apps and a GUI application, two NAnt tasks, two MSBuild tasks, multiple stylesheets - we will continue to enhance the docs and examples to make this all easier for you.

So, where to from here? Well, by all means please download the trial version of NCover 2.0.1 and send us your feedback. Obviously the new support forums are the ideal place for your questions, feature requests and anything you want to see improved further.

Enjoy! Oh yeah, and don't forget to encourage your coworkers / boss / accountant / significant other / postman etc. to purchase NCover 2.0 licenses if you don't already qualify for a free one... you know you want to...

NCoverExplorer... v1.4.0

2007-09-16T19:12:00.000+01:00

I've had this "quietly available" on the download page for a few weeks to gain feedback on any issues and with no known issues outstanding it seems appropriate to more publicly announce it.

A big change in NCoverExplorer 1.4.0 is that it is no longer published under the GPL license. I made this change for a number of reasons; amongst which was the concern that a number of companies have of deploying GPL based tools into their development process. This roadblock has been removed and should no longer prevent your companies from using this toolset.

My original choice of GPL was based on using the ICSharpCode 1.x texteditor control. I have now dropped that component in favour of using the superior (but commercial) Actipro SyntaxEditor control instead. The improvements are not only in stability and features but also visual as you can see below in the code highlighting screenshot. A splitter window is available as also shown.

So why should you upgrade to NCoverExplorer 1.4?

This release is less about adding new features (although there are some to be found) and more about stabilisation for the future. The core model code has been rewritten and better tested. There have been countless code coverage corner cases (say that five times in a row!) arise that should be better catered for. Better performance for .NET 2.0 users, lower resource usage, better error handling and some minor gui quirks addressed.

Some of the "internal" changes have spilled over externally. For instance the configuration file format used by NCoverExplorer.Console with /c has changed to support new features, although I have retained legacy compatibility for those who choose not to upgrade their build scripts immediately.

What about the new features? Well, the ones I did squeeze in were...

Support for the //pm (profiling a process by name) argument in NCover 1.5.8;
Copy command in the source code text area via right-click;
Print preview command in the File menu;
/fc argument for NCoverExplorer.Console to fail a build if the combined total coverage of all assemblies does not meet the minimum specified. This was the original behaviour of the /f command before it was changed to fail if any individual assembly does not meet the minimum;
Coverage exclusions now support the '?' wildcard and multiple '*' wildcards within the expression.
Report xml contains a "totalSeqPoints" attribute as requested for developers writing alternate xsl reports that need to calculate the excluded code at each level;
Update the NAnt/MSBuild task dlls to support the latest changes;
Redesigned the options dialog as shown below;

Downloads
Release Notes

CruiseControl.Net... Release 1.3.0 With Integration Queues

2007-06-22T20:12:00.001+01:00

Some of you may remember this post from last October about a patch I made for serialising builds in CruiseControl.Net. Owen Rogers was kind enough to give me the "keys to the kingdom" to get this integrated into the main CC.Net branch and as of today it is now officially released in CC.Net 1.3.0.

Having had several pair sessions watching Owen "fly" with Resharper was a great experience - many thanks Owen for the effort in getting the final release out as it can't be easy with new baby and work pressures!

As I said from the very beginning this particular iteration doesn't solve everyone's problems of complex build dependencies, nor was any attempt made to do so. However applying the 80/20 rule it appeared many people were like us with the simpler "this project should never build at the same time as this project" requirement. This implementation should do that just fine with a few bells and whistles such as viewing queue contents, controlling the build order and cancelling pending builds...

We have been using the nightly builds of 1.3 for quite a number of months now and it has proven solid from our experiences with it. There are of course further enhancements that can be made, but in discussion with Owen we froze it as is for people to get started with it and feedback their own thoughts. For instance one addition I would like to see is to identify in the CCTray queue which items are continuous triggers (checking for modifications) versus which are actual builds that will take place. Perhaps a simple "F" stamped into the icon corner for forced builds with additional information being passed in the xml request from the server about the integration request.

Documentation for how to use the integration queue feature is here.

NCover Problems / Fixes - Part 2

2007-04-19T18:34:00.000+01:00

In Part 1 of my "everything you wanted to know about NCover but were afraid to ask" series I covered the basics of how NCover works and profiler registration issues. Let's hit another of the most common problems:

Empty Coverage.xml File

This is quite an interesting one, as there are a number of possible causes. You perform your coverage run and don't see any errors in the console output. Yet when you open your coverage.xml file you see something like this from NCover 1.5.7.

<?xml version="1.0" encoding="utf-8"?>
<?xml-stylesheet href="coverage.xsl" type="text/xsl"?>
<!-- saved from url=(0022)http://www.ncover.org/ -->
<coverage profilerVersion="1.5.7 Beta" driverVersion="1.5.7.0" 
startTime="2007-04-19T18:38:47+01:00" measureTime="2007-04-19T18:38:47.921875+01:00" />

If using an earlier version of NCover the xml will be even less as there will be no attributes on the <coverage> node. So what has gone wrong?

Fundamentally it comes down to one thing - there were no debug symbol files found for assemblies that NCover thought it should be profiling. There are multiple reasons as to why this might be.

Debug Symbol Files

Ok, so not everyone knows what a .pdb file is and what purpose it serves. A program database (PDB) file holds debugging and project state information (Source: MSDN), such as line numbers, source code file information and sequence points. This is what NCover needs to produce in the coverage.xml file the line, column and filename information for each sequence point profiled in each method.

If you are compiling a windows or console application with a "Debug" configuration in VS.Net then by default a .pdb file will be produced for each of your assemblies. You can control this in the project properties dialog. Note that by default for "Release" builds no pdb file is produced - another common thing people forget.

For those of not using VS.Net, you can also produce these files using the command line compilers using the /debug option.

When NCover runs, it looks in the current directory of the loaded assembly for a matching debug file. If it doesn't find the .pdb for an assembly, you will see a "Failed to load debug symbols" message in the coverage.log file (more on the log in a future post). NCover then adds that assembly to it's internal list of ones to "skip" and will not bother attempting to profile it.

You can use this to your advantage. The less assemblies being profiled, the smaller the resulting coverage.xml file and the faster it will run. So rather than using the //a argument for NCover to specify assemblies (see below) you could simply delete the .pdb files from your testing bin folder prior to the coverage run for assemblies you do not want profiled.

Having ensured that you are correctly generating .pdb files in your output folder with the assemblies, what circumstances can lead to NCover saying it cannot find them?

- Your assembly was installed in the Global Assembly Cache (GAC)
- You are shadow copying assemblies such as when running nunit.

Profiling GAC Assemblies

When an assembly is installed into the GAC using gacutil it is physically copied to a physical folder location in:

%windir%\Assemblies\GAC\<simple name>\<version>_<culture or blank>_<public key token>

However what isn't so immediately obvious is that the .pdb file is not installed into that same folder by default. So when NCover runs and your assembly is loaded from the GAC, then NCover will not find the .pdb and will skip profiling for it. For the same reason you will have problems trying to use a debugger with that assembly as well.

So how can you get around this? Obviously uninstalling from the GAC is one option and certainly the easiest. I appreciate this isn't always the most convenient though.

Plan B is to move the mountain to Mohammed - and copy the .pdb files into the physical GAC folder. Unfortunately Microsoft didnt make that very easy for us but in typical fashion a number of people out there found a way. You can figure out the location using the pattern above - using the command line of course. Patrick Cauldwell has a blog entry and script for automating it with Powershell (Monad) here. Also Scott Colestock has a NAnt task for doing this here.

NUnit Shadow Copying

This scenario is one I only figured out today in response to an issue posted on the NCover forums here.

By default NUnit will shadow copy your assemblies to a shadow cache folder for running it's unit tests - no shock/horror hopefully in that statement. What isn't so obvious however is the behaviour if you compile your assemblies and .pdb files on one machine and then test them on another.

It appears that .NET does not copy the .pdb files to the shadow cache when they were compiled on another machine - only the assemblies.

So your unit tests still happily pass - but you get no coverage information. Yikes!

Why does it happily copy them on the same machine but not on a different one? I haven't had time to figure that out yet. My guess is some sort of security/signing type of issue but that is just pure speculation. If you know please tell me - I've put a thread on the nunit forums here.

What is the workaround? Either profile your assemblies on the same machine you compiled them on, or run nunit-console with the /noshadow argument. Works a treat.

Choosing Assemblies To Profile

NCover will by default profile all assemblies loaded into memory for which it finds .pdb files. If that path includes a lot of debug files for other assemblies that you aren't interested in getting coverage information for, then this will both harm your profiling performance and give you massive coverage files.

You may also intentionally want to exclude layers of your application - perhaps the presentation or unit test assemblies are deemed irrelevant to your coverage results. I exclude test assemblies from our projects as I feel they unfairly distort the overall coverage percentage. I appreciate others like to have them so they can spot how much of their unit test code was not run.

I mentioned above that a "quick and dirty" trick is to delete the .pdb files for assemblies you do not want coverage for. A more gentlemanly approach is to use the //a argument with NCover. However there are a couple of tricks to it.

Firstly, note that this is the AssemblyName, not the physical assembly file name! No path information, no .dll suffixes etc. A common mistake is to do something like this:

ncover.console.exe nunit-console.exe foo.tests.dll //a foo.dll

If you do this, then you will get an empty coverage file - as NCover did not find a matching AssemblyName. Almost certainly the correct command line for the above is:

ncover.console.exe nunit-console.exe foo.tests.dll //a foo

If you have multiple assemblies you want to report on, then they are separated by semi-colons.

ncover.console.exe nunit-console.exe foo.tests.dll //a foo;bar

Building this list up in a script dynamically can be a minor hassle - thankfully my NAnt and MSBuild tasks for NCover will take care of this for you. In fact, they allow you to use a physical file matching pattern to specify the assemblies to be profiled - the tasks then internally strip all the extraneous information and pass just what NCover expects.

NCover Problems / Fixes - Part 1

2007-04-18T20:21:00.000+01:00

From time to time I try to chip in with support on the NCover forums, not only with NCoverExplorer issues but NCover as well since they are so dependent. There are a bunch of issues which come up repeatedly, particularly for people new to using NCover.

In a perfect world (future release?) NCover itself could offer some more helpful error messages, but where is the fun in that? For now at least it is more a case of deciphering the clues from the console output and coverage.log files.

Hopefully this series of posts may help a few people and reduce the times I end up repeating myself on the forums, so win-win if it works for you. My examples will use NCover 1.5.7. You should also take a look at the NCover FAQ which is included in your installation. I've got a copy here too if that helps...

Profiled process terminated. Profiler connection not established.

Easily the most common problem people stumble on. As shown below your application starts (in this case unit tests), but after a delay of 60 seconds or so you get this message.

>E:\apps\NCover\ncover.console.exe E:\apps\nunit\nunit-console.exe MyTests.dll
NCover.Console v1.5.7 - Code Coverage Analysis for .NET - http://ncover.org
Copyright (c) 2004-2006 Peter Waldschmidt

Command: E:\apps\nunit\nunit-console.exe
Command Args: "MyTests.dll"
Working Directory:
Assemblies:
Coverage Xml: Coverage.Xml
Coverage Log: Coverage.Log

Waiting for profiled application to connect...
NUnit version 2.2.8
Copyright (C) 2002-2003 James W. Newkirk, Michael C. Two, Alexei A. Vorontsov, Charlie Poole.
Copyright (C) 2000-2003 Philip Craig.
All Rights Reserved.

OS Version: Microsoft Windows NT 5.1.2600 Service Pack 2 .NET Version: 2.0.50727.42
.
Tests run: 1, Failures: 0, Not run: 0, Time: 0.016 seconds

Connected
Profiled process terminated. Profiler connection not established.

Most likely this is due to one of two reasons I know of:

(1) You are profiling a windows service or IIS - this feature has been reported as currently broken.

(2) You are profiling an executable - but Coverlib.dll was not COM registered

Unfortunately the windows service/IIS testing fell down the cracks for 1.5.7 for those of you trying to use this feature. You can monitor some of the threads on it here and here. Personally for my applications I remove the need to profile services at all. By making the windows service a super lightweight wrapper and moving all the implementation out into classes that are more easily tested there is no need to unit test what remains.

I appreciate however there are scenarios for people where they do want coverage of services for integration testing etc - all I can say for now is that Peter is on the case. Let's now focus on the more common scenario for people trying to profile executables and offer some background info.

NCover Basics

So what's that registration bit about then - surely NCover is a .NET application? Sure, some of it is - but the really clever bit of NCover is the C++ implementation of ICorProfiler in Coverlib.dll, which is a callback class for use with the .NET Profiler API. The requirements of the API are that the callback class has to be COM registered and it cannot be written in managed code. By using some environment variables set within NCover before launching your process, the .NET profiler is able to be "switched on" to make the API callbacks while your process is running.

This is the heart of NCover - it monitors the loading of AppDomains and modules, intercepts the JIT execution of methods and injects it's own instructions to monitor the execution counts. This data is then pumped back to the "managed code" part of NCover, which munges the data together to produce the coverage.xml output you know and love for NCoverExplorer to manipulate.

So the "profiler connection not established" message indicates that there was some sort of problem with communication between NCover and the .NET profiler. As illustrated with the windows service problem above this message doesn't always mean that it was caused by a registry problem, but many times it is and it's worth double-checking first.

NCover Installation / XCopy Deployment

If you installed NCover using the MSI installer, then it will have automatically COM registered CoverLib.dll for you in "C:\Program Files\NCover\". If however, you are like many people intending to run NCover using an xcopy style deployment mechanism as with other tools such as NUnit, then you must remember about this requirement.

So let's say you do want to relocate NCover to a tools subfolder in a source controlled directory - what's the idiot-proof way to do it?

Firstly, make sure you copy all the NCover files from the installation folder above. Make sure you don't skip the Microsoft.VC80.CRT.manifest, MSVCP80.dll or MSVCR80.dll files. If you do skip them, then an attempt to register Coverlib.dll will fail on machines without these .dlls already in a path somewhere. You may find it works on your development machine with VS.Net installed, but then fails on your build server.

Secondly, make sure you run the NCover uninstall which will clean up the registry as well as the installed files. So now your machine should be back in "pristine" condition.

Options For Registering NCover

So having identified that when running NCover.Console.exe you need to ensure that Coverlib.dll is registered - what are your options?

1. If you stuck with the default NCover installation, have no need for locating it elsewhere and need only one version of NCover for all your applications. Congratulations - you shouldnt have seen the problem above! If you did, you should manually re-register it using regsvr32 or try running the installer again.

2. The NAnt/MSBuild tasks I provide automatically register and unregister Coverlib.dll on the fly under HKCU. So if using those you are already sorted.

3. If using the command line, NCover 1.5.7 now offers a //reg option. This will do the same as the tasks above of registering and unregistering on the fly.

4. You could invoke regsvr32 on it as part of your script. I suggest you unregister it as well afterwards in case you have multiple NCover versions on your machine to prevent any later confusion.

NCover With TestDriven.NET

I got myself caught out with this little gotcha recently and I'm sure I won't be the last!

As you likely know the excellent TestDriven.Net add-in by Jamie Cansdale includes a copy of (currently) NCover 1.3.3 and NCover 1.5.7 bundled with it. What you probably don't know is under which scenarios it runs which version of NCover and where this can go wrong.

The first thing TestDriven.Net does is check the registry to see if you have installed NCover yourself using the MSI installer. If there isn't then it knows to use one of the versions it has installed. If you have .NET 2.0 installed on your machine it will use the 1.5.x version, otherwise it will use NCover 1.3.3.

If however you do have NCover installed from the MSI, then TestDriven.NET assumes that you want to use that version. Jamie's rationale for this is simple - if you ended up with later versions of NCover on your machine than that supplied with TestDriven.NET then it should use that over it's own.

Where this can go wrong as it did for me is if for some reason you have unregistered NCover in that folder. TestDriven.NET makes the reasonable assumption that if you installed it and left it there then you have it registered there too. However as I dabble with testing and supporting multiple NCover versions all over my drives I had manually unregistered it without remembering to uninstall it. That mistake will then lead to error messages from TestDriven.NET about "Unable to create a coverage report". Either fully uninstall that NCover installation or register CoverLib.dll again in that folder to fix this. My thanks to Jamie for his excellent support!

TypeMock And NCover NAnt/MSBuild Tasks

TypeMock is a very powerful mock objects framework which no doubt some of you are using. Assuming you have purchased the commercial version then you will likely want to be running your tests with code coverage using NCover.

The one gotcha with this is that a requirement of the .NET profiling API is that there can only be one profiler callback registered on a machine when it runs. TypeMock takes care of this by managing the registry itself. However if you use my NAnt/MSBuild tasks then by default they want to "undo" the work of TypeMock by registering NCover, causing a problem.

The solution is available from the 1.3.6.11 build of my tasks in NCoverExplorer.Extras, the latest of which is available here. By adding a registerProfiler="false" attribute to your NAnt task or equivalent for MSBuild the default behaviour of the tasks to register in HKCU is turned off and control is able to be maintained by TypeMock.

That's enough for today. Part 2 will cover the mysterious blank coverage.xml file. Feedback/corrections welcomed - if you have an NCover problem your best bet is to head to the NCover forums.

NCoverExplorer... v1.3.6

2007-04-06T01:30:00.000+01:00

Some of you may already have noticed the long awaited release of NCover 1.5.7 yesterday - hoorah! That now being out of the way and in conjunction with some easing of my day job pressure, has made it a good time for me to finally rollout an NCoverExplorer update.

Note - if you have downloaded NCoverExplorer or TestDriven.Net over the last week or so you may want to do so again. I have been sneaking in fixes for issues right up until this blog posting. Check the downloads page to make sure you have the very latest build number.

This is predominantly a bug-fix release, addressing a range of issues that have accumulated over the last six months. Of utmost importance was improving the coverage result accuracy from either the merging of multiple coverage.xml files or multiple AppDomains within a single file. Some of you have experienced issues with the "optimisations" of removing noop sequence points that NCover 1.5.4, 1.5.5 and 1.5.7 apply in different ways. This resulted in duplicated nodes in the treeview, or lower than expected coverage with brackets being shown as not hit etc. My thanks to everyone who contributed to the testing of this with their coverage files/repro projects and patience while I got the kinks out.

On the new feature front, the biggest is a "Find Type" dialog (ctrl+F) allowing you to navigate to types by entering their name (supports * for a wildcard).

A change to previous behaviour is to the fail on minimum coverage option for NCoverExplorer.Console. Based on feedback this has now been changed so that any assembly not reaching the threshold will cause the build to fail, rather than total coverage.

All the other tweaks and fixes can be found in the release notes.

I have updated the NCoverExplorer Extras download containing the NAnt and MSBuild tasks. The <ncover> task now uses a reference counting style approach to the registration of coverlib.dll, which should get around issues reported on build servers of simultaneous build issues if you got unlucky with the timings.

The task now includes a registerProfiler optional attribute for those of you using TypeMock - you can set this to "false" to ensure the NCover task does not try to register CoverLib.dll itself.

There is an enhancement to the <nunitproject> task to support the appbase property for people using multiple testing folders (thanks to Garth Williams).

Another addition is a new <ndoc2> NAnt task in there for anyone using NDoc2 Alpha as I blogged about in this post. Documentation is available via the links on this page but anyone familiar with the <ndoc> task should be happy. I've used it for generating CHM documentation without any problems.

For any issues with the release or further enhancement suggestions, please post to the NCoverExplorer forums or send me an e-mail (Help -> Send Feedback).

A quick further mention on NCover 1.5.7. This is a mandatory upgrade for every NCover user as far as I am concerned. Peter Waldschmidt's major rewrite of the profiler core and the addition of an integration test suite sets up a great platform for future releases. I've been fortunate to be able to chip in here and there on this release, although arguably more nuisance than help at times to Peter I'm sure!

I am highly optimistic however that this is the most solid NCover beta to date and was worth the wait to get it right. Top stuff Peter and many thanks for all the effort!

Last but certainly not least, Jamie Cansdale has made available a new build of his excellent TestDriven.Net add-in. Amongst other goodies this now includes the latest build of NCoverExplorer, NCover 1.5.7 and Visual Studio Express SKU support. Right-clicky clicky goodness...

Downloads
Release Notes

VS2005 - When SGEN Doesn't Work...

2007-02-05T00:14:00.000Z

One of the features I was looking forward to making use of in .NET 2.0 was that of pre-generating the xml serialization assemblies, such as described in this post.

Using the sgen.exe tool is fairly straightforward, and there is an <SGen> MSBuild task for this purpose as well. However there is also one other option which not many people seem to have discussed the problem with that I could find.

If you look on the Build tab for your C# project you will see a "Generate serialization assembly" option with a badly aligned-combo box and a mysterious default option of "Auto" that is not described in the MSDN help - more attention to detail by Microsoft in VS2005.

Switching this combo to "On" and compiling a bunch of different projects I found each time that no xml serializer assembly was being produced. This had me rather irritated, and I know of others who had the same problem but no mention of why.

By looking at the MSBuild output with verbose detail you see a first hint as to what is going on:

Assembly 'C:\Foo\obj\Debug\Foo.dll' does not contain any types that can be serialized using XmlSerializer. Please use /verbose switch for more information.

How come? I can run sgen.exe from the command line on the same assembly and that works just fine producing output. Ok, lets take the command from the MSBuild output for sgen.exe and add /verbose to it...

Assembly 'C:\Foo\obj\Debug\Foo.dll' does not contain any types that can be serialized using XmlSerializer. Please use /verbose switch for more information.

WTF? I did add /verbose - fat lot of good it did me. What else is on that sgen.exe command line...

...sgen.exe /assembly:C:\Foo\obj\Debug\Foo.dll /proxytypes /reference:...

Aha - that /proxytypes option looks a bit interesting. The description in the MSDN documentation is "Generates serialization code only for the XML Web service proxy types."

An explanation at last - the property page option is effectively useless unless you are using web services. If you have other uses of xml serialization like many of us do then you have to resort to post-build tasks or scripting sgen.exe externally. All Microsoft had to do was change the combobox options to something like "Off", "ProxyTypes", "All" and this feature would have been far more useful.

Incidentally I also found if you remove the /proxytypes option the /verbose option starts working - quality.

[Update: 17th March 2006]
I should have posted the actual workaround I implemented, which I will now for at least my own purposes of reusing it. By adding the below extract into your .csproj file this will ensure that every time you compile the xml serialization assembly is regenerated for non web proxy classes.


  <!-- Grant - this bodge is necessary as Microsoft screwed up the xml serializer
    generation by limiting it to only web proxy classes.
    The one line change from the equivalent
    target in Microsoft.Common.targets is to set UseProxyTypes to false.
    -->
  <Target Name="GenerateSerializationAssembliesForAllTypes"
DependsOnTargets="AssignTargetPaths;Compile;ResolveKeySource"
Inputs="$(MSBuildAllProjects);@(IntermediateAssembly)"
Outputs="$(OutputPath)$(_SGenDllName)">
    <SGen BuildAssemblyName="$(TargetFileName)"
BuildAssemblyPath="$(OutputPath)" References="@(ReferencePath)"
ShouldGenerateSerializer="true" UseProxyTypes="false"
KeyContainer="$(KeyContainerName)" KeyFile="$(KeyOriginatorFile)"
DelaySign="$(DelaySign)" ToolPath="$(SGenToolPath)">
      <Output TaskParameter="SerializationAssembly"
ItemName="SerializationAssembly" />
    </SGen>
  </Target>
  <!-- <Target Name="BeforeBuild">
    </Target> -->
  <Target Name="AfterBuild"
DependsOnTargets="GenerateSerializationAssembliesForAllTypes">
  </Target>

Building Castles On Quicksand

2007-01-18T20:52:00.000Z

The projects I am involved in for my "day job" are moving to .NET 2.0 so lately it has been head down converting a huge array of wizards, templates, add-ins and of course the code itself. I also rewrote the core of NCoverExplorer over Christmas as part of a future 1.3.6 release but have had no further time for CC.Net, NCover or various other tools I had been tinkering with contributions for. Once things quieten down a bit at work I will be able to crank things up again.

Moving to .NET 2.0 at work has resulted in various amounts of pain thanks to bugs in the .Net framework, the IDE, the designers and the extensibility library. I've mentioned before I am not a fan of VS.Net 2005 and using it in anger these last few weeks has done nothing but lower my opinion. How some of these bugs or design decisions got through "quality control" is beyond belief. Actually I do have a theory on that but I will save that rant for another day.

The reason for this post however is to blog about my experiences with Sandcastle and generating help documentation for VS.Net 2005.

We have been using NDoc of course for our .NET 1.1 based documentation for VS.Net 2003. Unfortunately as was much documented and debated a while ago, Kevin Downs pulled the plug on this product. Part of the rationale given is Microsoft's announcement of Sandcastle, now up to it's third CTP release (Dec 2006) so I figured it might be mature enough to at least try it on for size.

Wrong, Wrong, Wrong.

If you haven't tried it and have been wondering on the sidelines like me - let me save yourselves hours of pain and frustration by suggesting you continue to wait. It is labelled as CTP but it is closer to the prototype end of the scale than the release candidate one - nowhere near ready for community usage, at least for our requirements.

So what got me so underwhelmed about it?

Well for starters the fact that Microsoft haven't yet included any form of command line or GUI means straight away you are into the world of evaluating and choosing other tools to actually make it work. Yes, I know you can argue choice is a good thing and I don't mean to denegrate in any way the effort people out there have put in to try to make up for this shortcoming by Microsoft. However sometimes you just want only one new moving part to add to your build process.

Next you need to install the VS.Net 2005 SDK to get the right Help2 compiler - it is no longer available as a separate small download like it was for VS.Net 2003 with VSHIK.

Then there is the list of "known issues" - mostly problems with the templates which mean your output is going to be kakked so you need to replace them. So that's usually four separate "things" to be downloaded, installed, configured and documented for your build server before you even try to configure a documentation project - straight away you are aware this is not going to be trivial.

So how do you automate Sandcastle? Well, there is a wiki which indicates at least some of the options contributed by the community - a batch script , an MSBuild script and several GUI options, the most popular being an NDoc lookalike called Sandcastle Help File Builder (SHFB). There are VS.Net add-ins as well although I'm after a tool that works as part of our automated build process - not locking up the IDE for hours thanks.

A dozen-step batch script didn't sound too promising to me, so I tried the MSBuild script by Anders Ljusberg first. Simple to deploy, a few tweaks for paths here and there, ran it and *boom* - first technical issue.

By default Sandcastle wants to connect to a web service on the internet - I believe to generate links to Microsoft content. Well that's just great if you live in the magical fluffy world where proxy servers don't exist, or if they do they take IE settings. Unfortunately where I work we have popup dialogs for authentication - which isn't catered for in any way I know of by the standard <system.net> configuration override trick. The "Start" page in Visual Studio.Net 2005 suffers from the same problem.

I couldn't see an obvious way to turn off that type of online linking using the MSBuild script (perhaps someone can tell me how) so that knocked it out of the equation for now. Incidentally I ran the script on my home PC which has no proxy server and hit a different issue, so it's clearly not the only problem you may face with using it.

Next I moved on to SHFB which does have an option in the GUI to set the external linking to "none". This looked far more promising having both a console and GUI version in NDoc style, some help etc - kudos to Eric Woodruff for his efforts. The price you pay over the MSBuild approach is less flexibility in terms of dynamically assigning assembly lists, version numbers etc since it works off configuration files. No major problem though as it is easy enough to knock up a NAnt or MSBuild task to generate the configuration file dynamically. First priority though is getting the output generated.

And that's where problem #2 came up - performance or rather appalling lack thereof. Generating a .CHM file for our 40 assembly application which used to take an hour under NDoc came in at 5 hours with Sandcastle. WTF? Ok, so leaving it running overnight is going to be the Sandcastle way of doing things. BTW, according to SHFB it took 4 minutes 55 seconds - rather wishful thinking!

There were a few problems with the output so I tweaked some settings (like saying I don't want namespace summaries as I dont want big red warnings everywhere) and ran it for VS.Net 2005 output on just three assemblies. Out popped the help - but still the big red warnings! Double checked my GUI configuration - show warnings are set to false. Hmmmm... Equally bad is that there is a horrid empty blue bar near the top of each page - I think it is expecting some sort of input to go in there but can't cope with it not being present.

So after all the research, downloads, configuration, bugs, trial and error I end up with something that takes hours to render and looks crap. Not quite the result I was after - yes I could try to fix the bugs but that's not what I set out to do. I just need to generate some documentation.

So what to do in the mean-time? I had tried the NDoc2005 project on SourceForge but found it completely unusable with bugs and the developers have clearly lost interest.

I then stumbled across something known as NDoc Alpha which appears to be the last work in progress build of the original NDoc developer Kevin Downs. It all sounds very promising on the web page in terms of support for generics and performance boosts.

I downloaded it and gave it a whirl for a .CHM file,and *boom* - 'access is denied' messages. NDoc Alpha copies your assemblies to a shadow folder so as not to lock the originals. The downside is if those assemblies have the read-only flag set it tries to do a delete from the shadow folder to clean up but doesn't check for the flag first. Now that is a pain - I wanted to build from my ClearCase view release folder which has those attributes set. Thankfully there is an option "UseAssemblyShadowCache" which when set to false disables this feature and hence works around the bug - nice.

Once I hacked that I was able to set NDoc Alpha off in motion again... and *boom* hit a series of big nasty exceptions which stopped documentation being generated. After much fiddling I figured out it was an xml tag that had the wrong attribute that VS.Net does not validate as an error. FYI it was <exception name="..."> instead of <exception cref="...">. In later iterations I also found it erroring where the tag had no attribute at all - i.e. <exception>. It appears that the new CHM documentor in this build of NDoc is far stricter than previous NDoc versions - not a bad thing, just it would be nice to have a more useful error message to figure out what is wrong with the tags.

Incidentally I had discovered that NDoc is spitting out Trace messages as it processes elements - that makes life a bit easier to find out what class it was processing when it failed. Fire up your favourite debug trace viewer such as from SysInternals and you hopefully get a hint as to either the method that failed or at least what once had processed successfully in the class.

Finally with all the error messages cleared NDoc Alpha had a clear runway. Boy is it quick! It took a mere 15 minutes to build the documentation from scratch and generate the .CHM file and a similar amount for the VS.Net 2005 version. Now that's just a complete no-brainer compared to Sandcastle. Sorry guys, you have a long, long way to go.

All that remains now is knocking up a new <ndoc2> NAnt/MSBuild task so I can generate the configuration file on the fly. Shouldn't be a major deal, and I will include the <ndoc2> task I write in NCoverExplorer.Extras.zip when I'm done for anyone interested.

The not inconsiderable negative of NDoc Alpha is that it is an end of line product - no source code is available to fix the bugs or anyone to offer support. It looks like Kevin has the last laugh on this for a while longer - his product is indeed still light years ahead of Sandcastle but the community are not going to be able to advance it further. That's frankly a damn shame as this is undoubtedly the best version of NDoc yet - after all the years of effort to let the source code waste away in the cupboard rather than live on in the community by others is a major downer... forgive the pun.

In the mid-long term then undoubtedly provided Microsoft deliver on the hype then they will be the product of choice - particularly for people not looking to use a commercial offering. For now though Sandcastle is a major thumbs down for me.

BTW - does anyone actually like the new "language filter" that is a part of the VS.Net 2005 help? You only work in one language but have to do half a dozen clicks to just see the C# examples by unselecting everything else first? Then if you get a page which doesn't have the language you are after, you go through another clickfest to see if there was an example in another language instead? If only Microsoft would make it a persisted option - setting a default language with an ordered list of alternatives if none of your defaults exist, how hard is that?

If anyone reading this feels I'm being too harsh or you wish to correct me please feel free to slap me round in the comments. My disclaimer is I'm under pressure to get stuff out the door so my "dive in and fix it" threshold is much lower than normal - using the current versions of Sandcastle and the related tools is not for people in a hurry from my experience.

Creating NUnit Projects Dynamically

2006-11-15T00:02:00.000Z

I referred in my Merging NCover Reports blog entry to a NAnt task I created a long while ago to resolve the issue of running multiple unit test assemblies in a single coverage run while using NUnit. By creating a .nunit project on the fly based on a wildcard pattern for test assemblies we avoid the hassles of manually maintaining it as the solution projects grow. For TestDriven.Net addicts who rarely run the NUnit GUI like myself this gives the best of both worlds.

Since a couple of people asked me to make it available I have now added both this and an MSBuild version to the respective NCoverExplorer.NAntTasks.dll and NCoverExplorer.MSBuildTasks.dll assemblies.

Download the latest NCoverExplorer.Extras.zip file to get the compiled assemblies, source code and documentation with examples. For a quick and dirty preview, the NAnt task looks something like this:

<nunitproject project="${outputdir}\MyApp.nunit" appConfig="${outputdir}\MyApp.exe.config">
     <fileset basedir="${outputdir}">
         <include name="MyApp.*.Tests.dll" />
     </fileset>
</nunitproject>

You can now change your NAnt task executing NUnit so that it is passed the name of the .nunit project file rather than the testing assembly inside some loop. This will give you a single xml file summarising the test runs. If you similarly do the same for running NCover you get a single coverage.xml file for your project.

I should add that this has only been used with the NUnit 2.2.x project versions - having not had a chance to look at the NUnit 2.4 betas myself as yet perhaps someone can let me know if they significantly changed the .nunit project file format?

NCoverExplorer... v1.3.5

2006-10-30T20:31:00.000Z

The latest build (1.3.5.1921) is now available for download from this site, which includes updates to both NCoverExplorer and the NCoverExplorer.Extras.zip file (for the stylesheets and NAnt/MSBuild tasks). I have held off officially releasing for a while so as to offer the usual "combined installation" option with the TestDriven.Net 2.0 RTM which Jamie Cansdale has just released - congratulations Jamie!

The leading new feature in this release as I blogged about a number of weeks ago is enhanced integration with NCover for generating NAnt scripts, MSBuild scripts and command lines. These should support the whole gamut of permutations of NCover versions (1.3.3 and 1.5.x), with or without my custom NAnt/MSBuild tasks. Of course you can also choose to just run NCover directly from the same screen, with the results being displayed immediately within NCoverExplorer. For people not using TestDriven.Net or needing to report coverage without using the VS.Net IDE this should hopefully prove a useful option.

Function coverage is another significant enhancement. For a while now NCoverExplorer has had "function coverage" as just showing the #visits incurred by each method. This has now been expanded to include a wider range of options to show percentages of methods covered on a class and/or number of methods unvisited.

There is also a new coverage report "Module/Class Function Summary" which can be used within NCoverExplorer or CruiseControl.Net (please upgrade your CC.Net stylesheets).

A new filter option of "Hide All Above Threshold" has been added to supplement the existing options of hiding unvisited or 100% covered nodes. For those of you who set a threshold for coverage below 100% this option should allow you to focus quickly on just the elements of the codebase that require your attention. Filtering and sorting options are now automatically reapplied when refreshing/reloading files in a session.

Sorting and filtering applies to reports either from within NCoverExplorer or optionally from the command line. NCover.Console.exe adds new /sort: and /filter: arguments. Both these and the /report: arguments will now take either numeric values or the enumeration text, as do the NAnt/MSBuild tasks. Type ncover.console /? to see the full set of options.

e.g.
ncover.console /h /report:ModuleClassFunctionSummary coverage.xml
or
ncover.console /h /report:5 coverage.xml

A significant amount of refactoring has been done to the NAnt and MSBuild tasks for NCover/NCoverExplorer to fix some bugs with various NCover versions. As part of this exercise I ended up renaming the assemblies and namespaces. My apologies to those of you who have to update your scripts but it made the documentation generation and maintenance easier for me. Speaking of which, at long last some documentation for these tasks is published outside of the source code. It is available both within the NCoverExplorer.Extras.zip and found here for MSBuild and here for NAnt. You will also find these links on the NCoverExplorer Help menu.

On top of all that there have been the usual swarm of important bug fixes (such as a fix for memory leaks) and minor tweaks - please read the Release Notes for more details. As with every release I encourage you to upgrade so as to trade in your old known bugs for some shiny new ones.

While NCover 1.5.5 was released a little while ago there are unfortunately some "showstopping" issues with it not only with .Net 1.0/1.1 but also when used with NCoverExplorer. I posted the details about the problem here on the NCover forum. The issues will impact the integrity of your coverage results but there is absolutely nothing I can do to workaround the issues - garbage in, garbage out I'm afraid. Until NCover 1.5.6 is released your best bet may be to stick with NCover 1.3.3 or NCover 1.5.4 or else you may possibly end up with results like this:

For any issues with the release or further enhancement suggestions, please post to the NCoverExplorer forums or send me an e-mail (Help -> Send Feedback).

Download TestDriven.Net 2.0 combined install

Download the latest NCoverExplorer versions from here.

Release Notes
FAQ
MSBuild Task Documentation
NAnt Task Documentation

CruiseControl.Net... Serialised Build Queues

2006-10-29T21:20:00.000Z

As a little break from NCoverExplorer development I started dabbling last weekend with the CruiseControl.Net source code. A disclaimer up front - this is all at the experimental stage and subject to further approval by the CC.Net project admins but it looks promising so far...

The main problem I am trying to resolve is the situation where you want to prevent two projects from running at the same time. Why would you want to do that? Perhaps you have resource constraints like a file system directory or a database the unit tests run against. For instance we have separate projects for continuous and daily builds - without a locking mechanism we likely get a failed build if they both run at the same time (one may do a "clean" while the other is trying to "build" etc).

A useful plugin was developed recently by Richard Hensley available from here to implement a locking mechanism which does help solve this. However it has a few limitations due to being a plugin which are best overcome by changing the CruiseControl.Net source code (in my opinion anyways). Various other implementations I have seen or even developed myself using mutexes or lock files all have similar issues.

What sort of limitations? As a build manager and developer I want visibility of the build queues, the ability to cancel pending items on the queue and to more granularly control the order that things get built.

My first stab of this is still a work in progress but looks like this...

1. You can optionally add a "queueName" attribute to each <project> node to force projects into a serialised integration queue. If you don't specify a queue the project will have it's own queue of the project name - effectively the same behaviour you have now.

2. You can also optionally add a "queuePriority" attribute to control the order of pending integrations on the queue. A priority of zero (default) just means build in the order placed on the queue.

3. A new optional panel in CCTray (see the demo gif below) to display the state of your queues across all polled servers. From CCTray you can right-click to cancel pending integrations provided they have not yet started.

4. A new web page plugin to allow web-based viewing of the queues and cancel pending integrations. Still to be developed.

Here's a little taster of what it looks like at the moment...

Comments, suggestions and feedback welcomed... there is a thread for this on the ccnet-devel Google newsgroup.

[Update: 22nd-Jun-07 This patch has been integrated into the CruiseControl.Net 1.3.0 release now publicly available]

NCoverExplorer... Merging NCover Reports

2006-10-12T21:06:00.000+01:00

There have been a couple of posts recently by Jeremy Miller and Haacked.com debating how best to structure unit test code. One of the arguments mentioned by a number of people against multiple unit test assemblies is the difficulty of merging multiple NCover coverage files on the build server.

To the people who posted comments along the lines of "it's just not possible"... well in actual fact it is, and here are a couple of ways to do it.

The first option for NUnit users is to use a .nunit project file listing your test assemblies. This requires only a single execution of NUnit and means that only one coverage file is produced in the first place, avoiding the problem altogether. I have a custom NAnt task that I wrote for dynamically creating the .nunit project file based on a file pattern just to avoid any maintenance - if anyone else is interested let me know and I will make it available.

The second option is to use NCoverExplorer. The console executable version is designed to be used on the build server. Included amongst it's abilities since v1.3.4 is the ability to merge raw NCover coverage files and save the result along with xml/html reports if required. In addition you can optionally apply coverage exclusions etc at the same time.

This can be run either directly from the command line or using the NAnt/MSBuild tasks available in the NCoverExplorer.Extras zip file.

So... to merge a bunch of suitably named coverage.xml files from a test coverage run into a single file using the command line it can be as simple as:

ncoverexplorer.console.exe *.coverage.xml /s:merged.coverage.xml

If you want to produce an actual report instead, just add the /r argument indicating the report type, and any combination of /x or /h to produce xml or html output as appropriate. Of course the xml output is designed to be merged into your CC.Net build file and transformed using the CC.Net stylesheets also in that Extras zip file. Use /? to see the full command line options.

The same operations can also be performed using the NAnt or MSBuild tasks, documentation for which is found here for NAnt and here for MSBuild.

[Update Nov 14th 2006: I have made available the NAnt and MSBuild tasks to dynamically create nunit project files as detailed here.]

BinarySerializer and Strings in .Net 1.1

2006-10-07T11:21:00.000+01:00

I've just verified that this problem does appear to have been rectified in .Net 2.0 so undoubtedly it is "old news" to many people but this one had us guessing for a while at work this week...

When using the .Net Remoting over tcp/binary with a "large" collection of objects (6,000) we found that our application was going into meltdown and taking over 6 minutes to return the results - yikes!

The actual population time of the custom collection from the DataReader was negligible - it was when it hit the "return" statement to leave the application server domain that the CPU utilisation shot up to 100% and the lag began. The objects being serialized had around 40 fields containing a few Int32 and DateTime data types with the bulk being strings.

As a colleague very funnily cracked at the time - asking the users to go Tools -> Options -> Tiers -> 2 Tiers was probably not going to be acceptable... ;-)

After much shagging around with a profiler and Reflector I eventually figured out that it was to do with the way the BinarySerializer works. It maintains an internal object table so as to ensure it only serializes each object in the graph once and avoid cyclic references no doubt. The problem is that it uses the hashcode of each object as a starting point for where to store it in that internal object table. If it finds the same object instance at that hash position it can return it, otherwise it adds another hash and tries again in the next "bucket" and so on until it finds a match (or a free space to put this object).

However when you have an object graph with lots of System.String field instances that all have the same value this design decision fell to bits. Such is the case when populating your objects using DataReader.GetString() which gives you a new string instance each time. As the hashcode for a string is based on it's contents you now get an exponential increase in hashing collisions the more rows you have in your result set.

As it turns out this is a known problem for which there is the dreaded hotfix dated from Dec 2004. I say "dreaded" as like most large companies out there we have absolutely no show of getting a hotfix deployed due to the logistics involved. When will Microsoft EVER get their act together over their .Net service pack strategy (i.e. more than once every 3 years would be a start!).

Our workaround? Well, as I have hinted at above what breaks the serializer is instances of strings having the same value. So why not have a cache of the string instances that you retrieve through when using DataReader.GetString()? That way the same value "xyz" is serialized only once instead of once per row - smaller object graph, faster serialization performance.

Sure enough after the change our total round trip performance dropped to under a couple of seconds for those 6,000 records (the serialization itself in under 0.2 seconds). Nothing like that great feeling on a Friday afternoon of having spanked a problem like this... That we hadn't really hit the problem previously was just luck - either screens had not many string columns in the objects; the string values were mostly unique; or the resultsets were not very large.

Another solution would have been to use ISerializable and take control ourselves, for which there are some good articles on CodeProject like this one. However the cost is it introduces another maintenance point in each of your entity classes. For large development teams in the early phases of a project with a continually evolving data model like my current one that's bound to go subtly wrong at some point. We may yet need to resort to that - but I would prefer to hold off until we know we need that extra few drops of performance!

As I said at the top this problem seems to have been rectified in .Net 2.0 from my quick testing this morning - had we been using that at work it would have avoided many hours of frustration this week. Then again as per my .Net 2.0 TreeView performance problem post we would likely have had some other problems to deal with. I guess this is why we get paid the big bucks right?

CCStatistics for CruiseControl.Net 1.1

2006-10-05T21:40:00.000+01:00

A few days ago CruiseControl.Net 1.1 was officially released. Thanks to all those responsible - it has become an entrenched daily tool for many of us. I've upgraded a couple of build servers so far - well worth doing for the performance improvements and statistics features alone. This also gave cause for me to knock up a supporting tool called "CCStatistics" - more on this below.

When upgrading there are a few things you need to change in your CC.Net project files that are not all listed in the release notes...

Remove any <publishExceptions> elements.
Change the format of your <weburl> links to jump to the latest report in CCTray. They should now be in the format below, unlike the query-string approach used previously:
http://localhost/ccnet/server/local/project/MYPROJECTNAME/ViewLatestBuildReport.aspx
If you want to use the new statistics functionality you need to add a <statistics /> node to your <publishers> section. Without it when you click on the "View Statistics" page you get a nasty exception being thrown.

Speaking of statistics, if you hadn't guessed already this is one of the more interesting additions for me. In my opinion it is a "0.9" version in terms of readiness for release - there are a number of bugs I found I have added to Jira and the documentation is incomplete or misleading in a couple of instances at the time I write this.

That said it has the potential to be a stonkingly good feature. Who doesn't want to see the most common reasons their build fails (when you have ClearCase performance issues like we do) or see their unit tests count, code coverage etc over time all on a single web page? No more endless drilling down into build log hyperlinks...

To use this feature you add a <statistics> node to your <publishers> section in the project file. I recommend you put this at the end of your publishers (after any <merge> element) so that way you can get additional custom statistics from files merged into the build log. When CC.Net runs the project it will then generate in your artifacts folder:

report.xml (not statistics.xml as the documentation says). Contains all the statistics for builds to date.
reports.bmp (which is actually a png - JIRA). A graphic illustrating the "TestCount" statistic across the build log cycle.
statistics.csv (JIRA). Intended to be an exportable alternative to report.xml - but it has garbage output in it currently due to a few bugs.

When you click on the "View Statistics" link from a build project page you will see the standard statistics on a page like the following:

The report is fairly basic in appearance but that should be easily changed by editing the statistics.xsl file. That is next on my list and once done I will contribute it for anyone interested (or feel free to beat me to it!). Strangely the reports.bmp file is not referenced in this stylesheet by default - so the graph produced is not actually being used yet as far as I can tell.

A minor limitation in the current release is that you cannot "replace" statistics with your own ones without editing the xsl. Not the end of the world but again I've posted a feature request on JIRA on ideas to make that more flexible. You can however add new statistics as mentioned here. So for instance if we want to add the code coverage totals for the build as a result of running NCoverExplorer we can do this:

<statistics>
<statisticList>
<firstMatch name="Coverage" xpath="//coverageReport/project/@coverage" />
</statisticList>
</statistics>

This will result in an additional column being added to the statistics page on the right-hand side. You could of course add other NCoverExplorer statistics as well such as lines of code etc.

There is a catch to all this - statistics will only be generated for future builds, not for your legacy ones. That is where my CCStatistics tool comes in. This will parse all your build log files for a project and produce an updated reports.xml, reports.bmp and statistics.csv file as though you had been running CruiseControl.Net 1.1 "forever".

There is another benefit to this tool I can see. Undoubtedly you will over time decide on additional statistics you would like to measure. Provided they are sourced from data that existed in historical build logs, you can regenerate all your statistics to date with the new data included to give you a much better picture of the progress of your project. If the data isn't present (say you just started merging NCoverExplorer code coverage) then the values will be blank.

It should be pretty trivial to use - drop the executable into your CruiseControl.Net\server folder, click on the Load button to list your CC.Net projects, choose which to recalculate statistics for and away it goes...

I've made all the source code available - perhaps the CruiseControl.Net developers might be interested in adding it to their solution and maintaining it? There is also a compiled executable against CC.Net build 1.1.0.2498 (aka CC.Net release 1.1.1). To use the source code just unzip into the same \project source code folder as the other CC.Net projects and add to ccnet.sln.

This was a very quick hack from a few hours work and has the usual "it works on my machine" proviso. There is no care or attention over internationalization and I have only ever used the 1.x versions of CruiseControl.Net. So use at your own "risk" - although as the only file access it does is reading logs and writing the couple of statistics files I don't think there's much to go badly wrong. The code itself has a few "hacks" so as to work without touching the original CC.Net source code - these may be cleaned up at a later date if the dependent objects are enhanced. Note also that this code results in the same "useless" .csv file - that bug is in CC.Net source and even if I did work around it obviously your future builds would still have incorrect lines being published.

If you notice any problems or have suggestions feel free to let me know. Note that it will backup your previous statistics files with a .old prefix so if you notice a problem after running it once you can revert.

Also thanks to Jamie Cansdale for his help today with a sticky issue I had (the NetReflector component only "works" if the assembly containing the type it is trying to instantiate is loaded in memory - just referencing the .dll containing it is not enough).

Download CCStatistics executable (Depends on CCNet 1.1.0.2498)
Download CCStatistics source code

[Update 14-Oct-06: I've pushed up a new version which includes progress bars and timings. When processing a big directory of large files CCStatistics may become unresponsive - don't panic as it will finish eventually. I have also updated the unittests.xsl stylesheet to fix the still outstanding issue in JIRA of test suite failures not showing up that I blogged about previously.]

[Update 22-Sep-07: Damon Carr has significantly updated this tool to work with the latest CC.Net (1.3) which you can can download here as mentioned in the comments below. Thanks also to Drew Noakes for his previous work for compatibility with CC.Net 1.2. Having "served its purpose" at the time for my needs I did not have the time to maintain it myself but am glad that others have found it useful enough to do so. Well done guys...]

.Net 2.0 TreeView Performance Problem

2006-09-21T23:09:00.000+01:00

No doubt some of you may have noticed some pretty poor performance in NCoverExplorer when running it under .Net 2.0. I've made some tweaks recently for the imminent NCoverExplorer 1.3.5 release which should improve things, but it still sucks big time in performance compared to .Net 1.1.

I thought that all things .Net 2.0 were supposed to be brighter, shinier and faster I hear you say? Anyone who has had the misfortune to use the dog turd that is the VS.Net 2005 IDE will happily put that "generalisation" to rest (I'm saving that rant for another post). It appears that sadly the same "Microsoft Improvement Programme" has been applied to the TreeView control and it's giving me the heebies.

The problem? I need to update the .Text of the nodes in the tree dynamically. I do this on a lazy loaded basis (mostly for speed reasons that worked under .Net 1.1). As many NCoverExplorer users will know, I offer a variety of "views" such as %, #unvisited nodes, function visits etc - all of which require updating the .Text property of the nodes as you expand them.

In .Net 2.0, Microsoft changed the internal implementation of the TreeView control - but not for the better for my requirements.

In .Net 1.1, calling .BeginUpdate/.EndUpdate around your updates to .Text did absolutely nothing in terms of performance gains. In .Net 2.0 however, you absolutely MUST wrap those same updates to .Text in this call, or else your performance really goes to hell and back (under the hood it generates literally millions of windows messages for a 10MB coverage file loaded). Sure enough this does improve performance considerably - but still nowhere near .Net 1.1 levels as you can see below.

You can download a little test app I knocked up from here. The results of loading around 15,000 nodes in a 1 x 25 x 25 x 25 hierarchy and expanding a node to see it's 25 children are shown. It's a standard TreeView with an override to OnBeforeExpand, which simply loops through the immediate child nodes only and updates the .Text with/without a BeginUpdate/EndUpdate wrapper.

Under .Net 1.1 (time to expand a node):
0.2 secs (using .BeginUpdate/.EndUpdate and updating .Text of the children)
0.2 secs (just updating .Text with no .BeginUpdate/.EndUpdate wrapper)
0.007 secs (just calling .BeginUpdate/.EndUpdate and not updating anything)

Looks pretty good right? Now the bad news:

Under .Net 2.0
1.1 secs (using .BeginUpdate/.EndUpdate and updating .Text of the children)
30.6 secs (just updating .Text with no .BeginUpdate/.EndUpdate wrapper)
1.1 secs (just calling .BeginUpdate/.EndUpdate and not updating anything)

So using .BeginUpdate/.EndUpdate (as you are now forced into under .Net 2.0 as you can see by the nightmare time of not using it) it is still around six times slower using .Net 2.0. That's just crap.

Interestingly with the third of those statistics above you can see that the problem is the .EndUpdate call, which in .Net 2.0 now does a whole lot of windows message blasting - regardless of whether there actually was anything updated as there wasn't in this case.

It is also obviously related to the size of the tree. Below is a graph showing the results with progressively 5, 10, 15, 20, 25, 30 and 35 child nodes under each node. Remember I am only updating the immediate child nodes, not every node in the tree!

It's pretty obvious that the .Net 1.1 implementation both performs and scales a heck of a lot better than .Net 2.0. If there is a reason for this garbage I'd love to hear it...

I'm open to suggestions from anyone else out there who has hit the same problem and figured out a solution. I started a thread on the Microsoft forums in desperation. I already lazy load so users should only incur the hit the first time they expand a node. However it's still pretty darn annoying to wait over a second every first node click to see just a couple of child nodes appear underneath!

NCoverExplorer... In The News

2006-09-13T23:32:00.000+01:00

Eric Sink just posted an interesting article on the usage of code coverage.
Scott Hanselman published his latest developer tools list - rather chuffed to make it on there!
Jim Holmes and James Avery near the end of their mammoth "Windows Developer Power Tools" book which I was delighted to contribute an article for earlier this year. Looking forward to the full read when it comes out around November...
Jay Flowers pushes out the first cut of CIFactory with an NCoverExplorer / NCover module.
Diedrik Krols over at Real Developer Network published a couple of examples of using NAnt with the <exec> task to merge multiple NCover coverage files and produce NCoverExplorer results for CruiseControl.Net

NCoverExplorer... v1.3.5 Beta

2006-08-20T23:25:00.000+01:00

This release is still a work in progress - you can read about all the new features in more detail on this thread on the NCover website.

The feature I want to highlight is the automation of NCover from NCoverExplorer:

Generate NAnt and MSBuild and NAnt scripts, with or without my custom tasks
Generate command lines for automation via batch files
Option to register CoverLib.dll before running NCover
Works against all major NCover versions (1.3.3, 1.5.x)
When run within the NCoverExplorer environment will display the console output and load the coverage results.

Click to see an animated example.

I've spent quite a bit of time helping people out on the NCover forums, often with variations of the same old issues of the command line, registering coverlib.dll, incorrect parameters, paths with spaces etc. Hopefully this should help people out with the drudge of first time command line syntax or MSBuild/NAnt scripts.

Not officially released yet as it needs some testing and feedback - hopefully will be a full NCover 1.3.5 release in a few weeks. Please use the thread mentioned above for feedback on any issues you find, or for the download links.

NCoverExplorer... Help Forums

2006-07-31T21:58:00.000+01:00

My thanks to Peter Waldschmidt for adding some NCoverExplorer forums to his NCover website. I had a comment today on my blog from Craig Andera who rightly raised the point that it would be nice for users to have some visibility on issues people have raised, when they get fixed etc.

I use FogBugz for my internal bug tracking and feature requests (thanks to Jamie Cansdale!) which is great for what it does. However it does have a limitation in that it is not publicly visible - so you guys can't tell if you are the 1st or the 50th person to have an issue or make a suggestion. With no real other public channel for communication other than my blog I agree it was long overdue for me to to do something about it.

So - please feel free to make your NCoverExplorer suggestions, tell me what works or doesn't - or indeed by special exemption talk about the All Black supremacy in the Bledisloe Cup over Australia (again) is permissible... (Rugby World Cups are not however).

In return I shall try to also post in there about what's on the horizon and of course continue to try to help those in need with NCoverExplorer (or indeed NCover) where I can as hopefully others will too...

NCoverExplorer Forums

VS.Net... Cancel A Failed Build

2006-07-22T06:47:00.000+01:00

I only recently came across this Visual Studio Hack by Michael Wood to fix one of the single most irritating problems with both Visual Studio 2003 and Visual Studio 2005.

[Rant] How Microsoft continue to screw this up is beyond me - do they ever actually use their own products outside of their single assembly demos showing drag/dropping of connections onto forms? Why on earth when you have a referenced project failing to compile would you want to continue trying to compile everything underneath, ending up with screens of irrelevant errors and a Task List that is completely useless? [/Rant]

This macro is an absolute must have in your IDE... just add to the MyMacros -> EnvironmentEvents module (you may already have the WithEvents declaration):

<system.contextstaticattribute()> _
Public WithEvents BuildEvents As EnvDTE.BuildEvents

Private Sub BuildEvents_OnBuildProjConfigDone( _
 ByVal Project As String, ByVal ProjectConfig As String, _
 ByVal Platform As String, ByVal SolutionConfig As String, _
 ByVal Success As Boolean) _
 Handles BuildEvents.OnBuildProjConfigDone

 If Success = False Then
   'The build failed...cancel any further builds.
   DTE.ExecuteCommand("Build.Cancel")
 End If

End Sub

NCoverExplorer... Module Thresholds

2006-07-22T06:18:00.000+01:00

I received an e-mail from a user trying out the new "per module coverage thresholds" functionality which highlighted some things not clear from my examples in the original post. The example I gave for MSBuild (the same comments apply to NAnt or using a .config file) was this:

<NCoverExplorer ...
ModuleThresholds="MyAssembly1.dll=30;MyAssembly2.dll=80" />

The three things to note are that:

Currently the module name match is case sensitive. I have fixed this in the source code for a future NCoverExplorer release. For now, make sure the names match exactly with those you see in the report.

There is no path information deliberately in these module names. The comparison done under the covers by NCoverExplorer is against the NCover names without the paths.

NCoverExplorer will fail a build (if using the fail minimum switch) if either the overall coverage or any of the individual module thresholds specified are not met.

My thanks to Max Palmer for the feedback. I have also added to the FAQ.

NAnt and MSBuild Tasks for NCover

2006-07-16T05:08:00.001+01:00

I received an e-mail this week from Jay Flowers asking if I would be interested in helping him put together NCover and NCoverExplorer plugins for CI Factory. The intent is to provide these as an alternative for the CoverageEye package he already includes.

I normally use the NAnt <exec> task for running NCover on our build servers, which does the job fine once you have it setup. There is unfortunately a bit of pain involved in truly dynamic scripts ensuring paths with spaces are handled correctly, building delimited lists of assemblies etc. The NCover command line in combination with NUnit can be a frustrating learning curve, not to mention it has changed between versions so even looking for answers in web blog articles can be confusing when they are out of date or "simplified".

To make this a little more painless for both Jay and anyone else interested I ended up creating NAnt and MSBuild tasks for NCover, supporting NCover 1.3.3 and 1.5.x to date.

I have now included these tasks in my existing NCoverExplorer NAnt/MSBuild task assemblies located in NCoverExplorer.Extras.zip, along with all the source code and example .build/.proj files.

Note: These tasks still require that NCover has it's CoverLib.dll COM registered correctly on your machine. You can always do this as part of your NAnt/MSBuild script prior to using these tasks if not using the default NCover MSI installation.

Here is an example of the NAnt <ncover> task at it's simplest for NCover 1.3.3 - ~~note the use of the version attribute. The <ncover> task will default to NCover 1.5.4 command line syntax if this is not present.~~

[Update 03 Sep 2006: As of build 1.3.5.1789 the <ncover> tasks are self registering without needing local admin rights. The version attribute is also now redundant - it will be picked up from the NCover.Console.exe assembly if not specified]

<ncover program="Tools\NCover\NCover.Console.exe"
commandLineExe="Tools\NUnit\nunit-console.exe"
commandLineArgs="MyApp.UnitTests.dll"
~~version="1.3.3"~~ />

A more complex NAnt example using NCover 1.5.4 showing additional NUnit arguments, customised logging and xml filenames, a coverage exclusion attribute (in the root namespace) and specific non-test assemblies to report coverage on:

<ncover program="Tools\NCover\NCover.Console.exe"
commandLineExe="Tools\NUnit\nunit-console.exe"
commandLineArgs="MyApp.UnitTests.dll /xml=MyApp.nunit.xml /labels /nologo"
workingDirectory="${build.output.dir}"
coverageFile="MyApp.ncover.xml"
logLevel="Verbose"
logFile="MyApp.ncover.log"
excludeAttributes="CoverageExcludeAttribute">
<assemblies basedir="${build.output.dir}">
<include name="MyApp*.dll" />
<exclude name="MyApp.UnitTests.dll" />
</assemblies>
</ncover>

Here is a simple MSBuild <NCover> task example:

<UsingTask TaskName="MSBuild.NCoverExplorer.Tasks.NCover"
AssemblyFile="MSBuild.NCoverExplorer.Tasks.dll" />
<NCover ToolPath="Tools\NCover\"
CommandLineExe="Tools\NUnit\nunit-console.exe"
CommandLineArgs="MyApp.UnitTests.dll" />

I did manage to confirm a few bugs/unfinished features in NCover 1.5.4 itself while giving this a test, so where possible I have either worked around or excluded these features until they are one day resolved. FYI, they involved the //q and //h command line arguments, which you should avoid using!

Feel free to leave a comment or drop me an e-mail if you find these tasks useful (or find a bug of course!).

Download NCoverExplorer.Extras.zip (CC.Net/NAnt/MSBuild)

[Update: I'm more than happy to answer questions for people adding these into their builds. However rather than using the blog comments can you please put your query in the NCover or NCoverExplorer forums as I blogged about here.]

[Update 03 Sep 2006: NAnt and MSBuild tasks have been updated and improved. See this thread for the changes listed in build 1.3.5.1789 or the 1.3.5 release notes. Documentation can now be found here for MSBuild and here for NAnt (or in the zip file).]

[Update 07 Sep 2006: Bug-fix update for the NCover tasks where multiple arguments specified in the command line. Build 1.3.5.1797 now available at the above links.]

[Update 01 Oct 2006: Updates to the NCoverExplorer tasks to support a new regular expression feature for coverage exclusions and improved documentation. Build 1.3.5.1861 now available. NOTE you must also upgrade your NCoverExplorer version to the latest 1.3.5 beta.]

NCoverExplorer... v1.3.4

2006-07-12T08:56:00.000+01:00

It's been a little while since the 1.3.3 release, but NCoverExplorer overload meant I had to take a break to keep some sanity. This release combines some important bugfixes with a small number of new features.

Firstly the new features:

On the toolbar of the NCoverExplorer GUI you will find four new buttons for navigating unvisited lines and classes. These have shortcut keys of 'N'/'P' (or forward/back mouse buttons) for the next/previous unvisited line in the current class. Use 'Ctrl+N'/'Ctrl+P' for next/previous unvisited class in this namespace (or Ctrl + forward/back mouse buttons). Thanks Jamie for the mouse button suggestion, it rocks!

You can use NCoverExplorer.Console.exe to merge and save your NCover coverage.xml files. Coverage exclusions will be applied prior to the save of the consolidated xml file. Wildcards are supported for the coverage filenames, so for instance you can do this:

ncoverexplorer.console.exe *.ncover.xml /s:coverage.merged.xml

If using the NAnt or MSBuild tasks from the latest NCoverExplorer.Extras.zip, you can add a new attribute "mergeFileName" (NAnt) or "MergeFileName" (MSBuild) containing the filename for the results.
Module level coverage thresholds are now possible using NCoverExplorer.Console.exe, rather than just a single threshold for your project. This is for people wanting more granular control where specific assemblies are deemed unrealistic to attain the same coverage as your overall goal, without seeing "failure" forever on coverage reports.

For NAnt, add this section within your <ncoverexplorer> task:

<ncoverexplorer ...
    <moduleThresholds>
        <moduleThreshold moduleName="MyAssembly1.dll" satisfactoryCoverage="30" />
        <moduleThreshold moduleName="MyAssembly2.dll" satisfactoryCoverage="80" />
    </moduleThresholds>
<ncoverexplorer>

For MSBuild, you can add this attribute to the <NCoverExplorer> task:

<NCoverExplorer ...
     ModuleThresholds="MyAssembly1.dll=30;MyAssembly2.dll=80" />

For complete examples refer to "example.build" and "example.proj" within the NCoverExplorer.Extras.zip file. If you do not want to use the NAnt/MSBuild tasks then you can use the command line with the /c option for a config file. The syntax is the same as for the NAnt <ncoverexplorer> task above - see the ConsoleExample.config file in your NCoverExplorer folder.
There is a new code coverage report, which drills down to classes per namespace per module. This is my "favourite" report we use on our CruiseControl.Net build servers. To generate this report the magic number is "4" for your report type attribute in the command line, NAnt and MSBuild tasks.

Updates to the CruiseControl.Net build summary stylesheet, so that it lists all of the assemblies, their coverage and whether they passed/failed as well as the overall project coverage. Note that this stylesheet now has a dependency on a transparent.gif file being placed in your CruiseControl.Net images folder. See the readme in NCoverExplorer.Extras.zip for details.

Added the ability to change source code paths for a loaded coverage file in the NCoverExplorer GUI. Sometimes you will be generating coverage files on another machine (such as a build server) that you want to browse - this feature allows you to override the embedded location to find the source code files in.

In terms of bug fixes, the most important that I blogged about recently is a solution for an issue when merging multiple coverage files. There are also fixes for memory leaks and various other minor GUI issues - see the release notes for full details.

As always, please drop me an e-mail through Help->Send Feedback if you have found any bugs or suggestions.

One final point - no doubt most people reading this have heard about Jamie Cansdale "going commercial" with TestDriven.Net. Jamie has been out there upgrading and supporting this great add-in for many years, for which many of us have enjoyed the benefits of "for free". Having had a small taste of what is involved in the last six months with NCoverExplorer you quickly gain an appreciation of just how much of your "life" supporting and enhancing these tools can consume! I wish him all the best in his opportunity to gain a return for those efforts - his continual innovation with TestDriven.Net continues to help fill the many gaps in the Microsoft "out of the box" tools to make for a far more productive development experience.

Download TestDriven.Net 2.0 combined install

Download the latest NCoverExplorer versions from here.

Release Notes
FAQ

NCoverExplorer... Merging Coverage Files

2006-07-02T14:09:00.000+01:00

Several users have reported issues when merging coverage xml files - the merge is aborted with the following dialog:

In response to this post on the NCover forum I explained why this occurs, without being able to provide an immediate workaround still using the merge feature.

I have just modified the way I process the xml files to workaround this problem and will include this fix in the NCoverExplorer 1.3.4 release which I will push out in the next week or two.

While on the subject of merging - I have also enhanced the NCoverExplorer.Console.exe and the NAnt/MSBuild tasks to now allow the automated saving of the results of merging multiple coverage files.

For anyone who can't wait for the official release please drop me an e-mail (use Help-> Send Feedback in NCoverExplorer) and I will give you a link to the current build. Any early feedback to identify any issues with these features or the others I have added for 1.3.4 would be appreciated.

Missing NUnit Fixture Errors In CC.Net

2006-04-27T20:43:00.000+01:00

I've noticed a few times that when we have an error in our test fixture teardowns at work that the CruiseControl.Net build report page didn't say what the cause was.

It correctly identifies an error return code from nunit-console, however it is not listed as a "error" on the build report or on the NUnit Details page since it isn't actually a specific test that failed. Rather confusing at the time...

I've tweaked the unittests.xsl stylesheet for CruiseControl.Net to fix this problem, and now get the test-suite failure message listed on the front build report page. If you are interested you can download it from here. I've also submitted it to the CC.Net developers group so hopefully it may be included in a future release.

[Update 14-Oct-06 - A new version of this stylesheet for CruiseControl.Net 1.1 is available from here (rename it to unittests.xsl). There is no real difference I saw - the CC.Net developer refactored their stylesheet to clean it up so I just reapplied the changes over the top. It is also listed in JIRA]

Illegal Characters & NCover Versions

2006-04-07T21:06:00.000+01:00

Some people are reporting problems with NCoverExplorer which are caused by an older version of NCover - that is NCover version 1.4.6. The problem occurs when they try to drill down into the source code within the GUI.

Please do NOT use NCover 1.4.6. The problem is within the coverage.xml file created - the filename attributes end up with extraneous characters appended (something like ''). I may try to strip this out myself in a future NCoverExplorer version, however as Peter does not even list this version for download nowadays I would rather people were forced to use a more "supported" NCover download.

So to clarify - which version of NCover should you use?

If you do not have the .Net 2.0 framework installed on your machine then you should use NCover 1.3.3. This is exactly what we use at work every day without any problems.

If you want to profile a .Net 2.0 application, then you must use one of the latest NCover 1.5.x betas. The very latest 1.5.4 beta does have some known issues (see the NCover forums - in particular here, here and here). Peter has had more important things to think about recently (congrats by the way mate!) but he has indicated the next release with some bug fixes will be coming out soon.

If you want to profile a .Net 1.1 application but happen to have .Net 2.0 installed then you can use either version of NCover. I'm sure Peter would much rather people were using 1.5.x than 1.3.3 so once his next beta release comes out that would be my preference, particularly for the coverage exclusion attribute support.

All supported NCover versions can be downloaded from NCover.org.

One final point - I see additional confusion coming from the two NCover projects out there. The NCover we want you all to be using is from NCover.org - unfortunately all too easily confused with another "inactive" project at SourceForge. Irritating I'm sure for all concerned that the two projects have the same name - just remember NCover.org and you can't go wrong.

NCoverExplorer... v1.3.3

2006-04-04T20:45:00.000+01:00

Another stack of new features in the latest generation of NCoverExplorer. I've added summary html/xml reporting, filtering capabilities, basic code metrics and very significantly a console version for automated build usage.

Html reporting as I posted about here has been requested by a number of users, including myself! For this release I have implemented summary reports since from the feedback I received they were of most interest. From either the GUI or the the console application you can now generate one of three reports:

Module Summary
Namespace Summary
Module and Namespace Summary

As you can see from this dialog you have a choice of output either directly to html or to xml (the latter having an associated stylesheet for rendering html).

An example of the module summary report is shown here. Note the inclusion of some basic statistics at the top right. You can also see that our "satisfactory coverage threshold" is displayed at the top (90%) and that any graph bars exceeding this threshold (but not 100%) have the uncovered code shown in yellow. So a glance at the report for the red bars will tell you which modules or namespaces are possibly in need of investigation.

Statistics are also available within the NCoverExplorer GUI. Note that these are derived from the NCover coverage report, not the source code. The totals deliberately ignore any "excluded" code.

Filtering offers a way of simplifying down the coverage tree without affecting the coverage totals (unlike "excluding" which recalculates the totals). There are two filters available - hiding all 100% covered nodes (for when you want to focus on the todo list), and hiding all 0%/unvisited nodes (for when unit testing isolated functionality).

My thanks to Lutz Roeder for the public donation of his excellent CommandBar code. You can see the impact in this in the new look to the menus with images and the new Toolbar shown below.

Including nodes that were marked as "excluded" is also now possible. You can think of this as an "undo" feature to override either manual exclusions or those defined in the options dialog. Including a node will also recalculate the related coverage statistics.

NCoverExplorer.Console came about from two features on my list. One of these from Matt Hargett of imeem.com was to enable failing a build if coverage does not meet a particular threshold.

The second item was the ability to produce xml and html coverage summary reports in an automated fashion. As anyone who has used NCover on large projects will testify, coverage.xml files can be very large. Rendering these using xsl while being an elegant and flexible solution is just way too slow, particularly when the coverage file is many megabytes. Merging the coverage.xml file(s) into your CruiseControl.Net buildlog.xml files also slows down the rendering of every single page for that build on the CCNet web server.

My solution to this is based on the assumption that I do not want to see line level coverage information on the build server. I just want to see a summary of the project, modules and perhaps down to namespace level of coverage. By adding NCoverExplorer.Console.exe to the build pipeline we can transform the NCover results very quickly into a tiny summary xml file. With the appropriate stylesheets we can now render coverage summary reports in CCNet in a fraction of the time previously taken.

All of the options available in the GUI in terms of exclusions, thresholds, project name, output as xml and/or html are available.

I have also developed an additional stylesheet for use on the Build Report page of CCNet as shown below:

Since these are just stylesheets transforming the NCoverExplorer xml file they can be customised to your own tastes if required.

The final piece of the puzzle is to automate through NAnt and MSBuild the creation of the xml/html reports. You can of course just use an <exec> task, however I have put together some custom tasks for both tools which offer a bit more flexibility. These are available for download in the NCoverExplorer.Extras zip file at the bottom of this post along with some included examples and the necessary stylesheets for CruiseControl.Net.

If anyone requires it I may put up a blog entry walking through such configuration or perhaps see about getting it documented on the CruiseControl.Net wiki. It is fairly straightforward though:

Copy the NCoverExplorer.xsl and NCoverExplorerSummary.xsl stylsheets from the NCoverExplorer.Extras zip file to your CruiseControl.Net\webdashboard\xsl folder
Edit your CruiseControl.Net\webdashboard\dashboard.config file to point to these stylesheets. I have included an example dashboard.config in the .zip
Modify your NAnt or MSBuild script. As stated you can either execute the console application directly with command line arguments or use my custom tasks. For instance to use the custom NAnt task copy NAnt.NCoverExplorer.Tasks.dll to your NAnt\bin\Tasks\net folder and follow the example syntax contained within "example.build".
Modify your CruiseControl.Net project file (ccnet.config) to use the merge publisher to include the xml output from NCoverExplorer.

For my own sanity, other than any bug-fixes I am intending to put a hold on further development for a little while. This has consumed every evening and weekend for the last couple of months so it's time for a break. Any donations could of course help rekindle my interest ;-)

As always I welcome any feedback. Enjoy!

Download TestDriven.Net 2.0.1578 combined install

Download the latest NCoverExplorer versions from here.

Release Notes
FAQ

NCoverExplorer... Want To Contribute?

2006-03-18T14:49:00.000Z

Any amount is appreciated - read more...

[Note - originally I tried creating this blog entry directly in here but Blogger wouldn't post form data correctly - my apologies for the indirection.]

HTML Coverage Reporting

2006-03-16T12:56:00.000Z

I've had a couple of users ask whether NCoverExplorer might ever offer the ability to generate HTML reports of code coverage. An example of the potential sort of output (including the source code highlighted) is shown by Clover.

I'm interested to know just how much demand there is for this feature. It looks pretty but personally I doubt I would make use of this particular report - html pages by their nature are static and the hundreds or thousands of pages generated are virtually guaranteed to be out of date immediately as the code changes. I'm open to being educated though - what usage would you make of such a feature if NCoverExplorer offered it, or is it just a "nice to have"?

The one type of reporting I have found useful is the top summary section produced by the coverage.xsl that comes with NCover. The downside of the XSL approach is it is just way too slow on large coverage files. It could be fairly easy for NCoverExplorer instead to produce a static HTML summary page showing the graph percentages. This report could then be included as a CruiseControl build artifact (if I make a console driven NCoverExplorer version) or via e-mail.

Does this have any appeal? If so, what sort of reports would you like to see (e.g. assembly coverage summary only or namespace/class summary). How detailed and customisable do the report(s) need to be?

All input for and against welcomed - either in the blog comments, or post your thoughts in the TestDriven.Net users group in this thread.

NCoverExplorer... v1.3.2

2006-03-14T19:43:00.000Z

Plenty of "must have" features that have been on people's wish lists in this release and a couple of irritating bugs fixed so I can't recommend highly enough that people upgrade. Download details at the bottom of the post.

Merging coverage files is the first of the "killer" features, offering improved integration with TestDriven.Net. Some of you may have noticed the inability to produce coverage results across a VS.Net solution or with multiple fixtures selected in the solution explorer window. The latest release of TestDriven.Net now supplies multiple coverage file names to NCoverExplorer which will merge and present the consolidated results. You can also manually merge multiple coverage files within NCoverExplorer via a number of ways, such as a "File->Merge" menu option or drag/drop.

Saving of the consolidated coverage results is now possible using the new "File->Save/Save As" options. Note that saving will remove any excluded nodes (see below) and merges together any duplicated visit count sections from the new coverage.xml file. This means you will likely find the resulting file much smaller than the originals.

Coverage exclusions are my personal favourite new feature. This offers a way for NCoverExplorer to automatically remove from the coverage tree items not relevant to your analysis when a coverage file is loaded - such as unit test fixtures, third party libraries etc. In the screenshot shown you can see the default exclusions I have included with NCoverExplorer - for instance any assemblies ending in ".Tests" will be excluded. You can add exclusions at the class, assembly and namespace level. I have also added support for the NCover 1.5.x feature of "exclusion attributes" if they exist in the coverage.xml file (although NCover still has some kinks to be worked out in a future release before you can rely on this feature).

All excluded items are then added to a special Excluded bin for the parent node as shown in the screenshot below. The source code for excluded methods will be highlighted in grey. You can also manually exclude items on the fly via a right-click "Exclude from Results" menu which replaces the previous "Remove from Results" option.

Source code tabs started out as a solution to a problem a few users reported with highlighting, particularly to do with partial classes. I believe I have resolved this once and for all now with the introduction of this feature. If you click on a partial class in the coverage tree, a tab is opened for every source code file making up that class with all the code highlighted as shown here.

The tabs also offer a way of quickly navigating around the coverage tree as selecting a tab will select the corresponding class node.

Themes came about as an answer to another much requested feature of more customisation of the GUI to include fonts and background colours. A "theme" is a group of settings covering most aspects of the appearance of the coverage tree, statistics pane and source code highlighting.

You can of course add your own themes as shown in the screenshots on this blog entry. Themes are easily toggled between using the new View->Themes menu.

Sorting and Coverage options came from a couple of user feature requests as shown here.

These allow NCoverExplorer to display function coverage or focus on methods with the most unvisited code.

My thanks to Trevor Hall for his time taken to suggest and test not only these features but also a number of others slated for a future release (after I take a break!).

Thanks also to Jamie and others who have been kind enough to test the beta versions and provide invaluable feedback.

There are numerous other minor usability tweaks and enhancements I haven't mentioned here that you can read about in the release notes or discover for yourselves. I look forward to any feedback!

Download TestDriven.Net 2.0.1545 combined install

Download the latest NCoverExplorer versions from here.

Release Notes
FAQ

NCoverExplorer... Partial Classes Grief

2006-03-03T00:13:00.000Z

I've had a couple of reports of issues with NCoverExplorer when used with partial classes. Thanks to Matthias Paetzold with his excellent repro I've figured out the cause.

You should only be experiencing this particular issue if you split your class variable declarations (with initial value assignments inline) into a different partial class file from that containing a constructor.

NCover groups all those lines of code together for each constructor - in effect creating something I call "partial methods" (that's not exactly the first words I used when I saw this issue but you get the idea!)

I'm hoping to get a fix in my NCoverExplorer 1.3.2 release out early next week to coincide with another TestDriven.Net public beta from Jamie. There's plenty of new NCoverExplorer features including:

coverage exclusions - automatically remove test assemblies/classes when a coverage file is loaded;
themes - greater GUI customisation and ability to save your own appearance combinations;
alternate display options - such as #unvisited sequence points or function coverage;
tree sorting capability - by coverage, unvisited sequence points etc;

NCoverExplorer... Updated v1.3.1

2006-02-16T22:05:00.000Z

Apologies to any of the "early adopters" of my latest build who downloaded yesterday and may have hit an issue. I have just pushed a fixed build online now along with TestDriven.Net build 2.0.1438.

The problem only arose when NCoverExplorer tried to highlight a multiline statement with a blank line in the middle. Not an everyday occurrence but certainly one user came across it (my thanks to Trevor Hall) - the code below would blow up fairly spectacularly with the previous v1.3.0 build:

To anyone wondering why this didn't occur with NCoverExplorer v1.3.0 or earlier - it's because I put in a number of workarounds for the otherwise brilliant ICSharp.TextEditor control in this release to improve the highlighting. The TextEditor control appears to be designed to highlight on a "word" basis for a line, whereas I want to highlight absolute positions as reported by NCover. Unfortunately it's word-based approximation sometimes resulted in misleading highlights on screen.

Fingers crossed this is the last of the tweaks on this issue... but as always if you have a problem feel free to report it.

NCoverExplorer... v1.3.1

2006-02-15T22:59:00.000Z

A ton of new features in the latest release, and a few bugs fixed ranging from minor annoyance (code highlighting) to downright critical for some (supporting .Net 2.0 features like partial classes and generics - thanks to Jamie Cansdale & Andreas Hofer in particular for pointing out the issues!).

As per usual you can download this version with the just released build (2.0.1435) of TestDriven.Net or as a standalone product via the links at the bottom of this post.
[Update] A post-release bug fix implemented on Feb 16th, updated on this site and available with TestDriven.Net build 2.0.1438.

What's New? Loads... more than I'm going to mention here (see the release notes for the full list). Let's look at the options dialog:

Apart from the new tabbed appearance there are some other new options to note here:

Namespaces will now be "flattened" by default which is the same appearance as Reflector or the VS.Net 2005 class viewer (see screenshot below).
For the VB.Net crowd out there you can now exclude the "My" namespace by default which NCover includes in the results, distorting your coverage percentages.
An optional threshold for the number of unvisited lines in a method. Acts in a similar way to the existing % threshold feature but just another way of convincing yourself things aren't as bad as they may first seem...

Probably the most requested feature I have had is the ability to customise the colours - so here you go! There is more to come in the future on this but for now you can change:

Moving onto the display itself, you can see some new menu options in the tree view and the default view of a "Flat" namespace hierarchy below:

The two new right-click menu options are:

Remove from results - does what it says on the tin. This option is context aware so you can use it to "prune" your coverage tree - be it to remove a unit test assembly as shown above or exclude a class. After the remove action is selected NCoverExplorer recalculates the percentages in the tree.
Collapse All - collapses every single coverage node in the tree.

Next is the statistics pane. This has had quite a revamp (my thanks to Andrzej Kaczmarczyk for his suggestions). Clicking on a class node now produces a summary of all the methods and the number of times they were executed (shown below). The columns are now sortable as well. This offers a rudimentary form of code profiling analysis.

The final new feature I think worth highlighting is that by default NCoverExplorer now tries to restore your cursor position / current selected node when reloading a coverage file. So when used in conjunction with the superb TestDriven.Net "Test With... Coverage" feature you no longer need to keep re-expanding the nodes to get to where you last were...

I look forward to your feedback with any additional suggestions, comments... or even dare I suggest new bugs! This release has made quite a dent in the features todo list but there's still a number of things on it yet...

Also very much worth noting - the latest NCover beta release 1.5.3 is now available - highly recommended for the bug fixes alone. Thanks Peter!

Download the latest NCoverExplorer versions from here.

Release Notes
FAQ

NCoverExplorer... Feedback

2006-02-07T20:55:00.000Z

As promised the latest build of TestDriven.Net with NCoverExplorer 1.3 included is now available... with this build of TestDriven.Net it will re-use the same NCoverExplorer instance each time you choose "Test with... Coverage".

We have also been trialling FogBugz as a way of tracking features, bugs and questions from everyone for both TestDriven.Net and NCoverExplorer. So far it looks pretty good - ease of use through simplicity is obviously it's goal and it does that well.

If you do have anything you want to report to us, the url (for the next 40 days during the trial) is here

NCoverExplorer... v1.3

2006-02-06T20:36:00.000Z

I've just finished putting the finishing touches on the latest release. If you want to download it straight away, you can find it in the locations at the bottom of this post. I believe Jamie Cansdale will be announcing availability of the next release of TestDriven.Net within the next 24 hours or so with this bundled...

What's new? Well as you can see in the screenshot below there's a couple of changes in the treeview. For a start, there is now a new root level node representing the coverage file, giving you a total for coverage of all it's children. In addition by default the coverage is now grouped by module (usually the same as the assembly). Great for large projects...

From a code navigation viewpoint a couple of convenience features have been added. The "Expand Covered" option gives you a quick way to open up all nodes with coverage - useful for if you have just executed TestDriven.Net with "Test with... Coverage" running a small subset of tests.

You can also see an "Edit in VS.Net" option - you can right-click either on the tree or within the source code pane and have your cursor appear at the same place in the source code in your Visual Studio.Net IDE...

A number of other tweaks and enhancements were made (such as reusing the same NCoverExplorer window when run from TestDriven.Net) - you can read the full details in the release notes.

A quick note of thanks to everyone who has blogged, e-mailed or otherwise commented and supported NCoverExplorer. I've got a load of more features on my list to come over the next few weeks (time permitting) - any other ideas are also welcomed either via the blog, the TestDriven.NET Users group
[Update] or preferably FogBugz during our trial over the next 40 days.

Download the latest NCoverExplorer versions from here.

TestDriven.Net Now Supports NCover 1.3.3

2006-02-01T18:40:00.000Z

As I hinted at previously Jamie Cansdale has just announced public availability of a new build which supports NCover 1.3.3 - good news for those of us who are still ploughing the BCL 1.1 fields without 2.0 installed. Jamie is clearly setting dangerous precedents in customer service by way of his superb "apology" - suggestions welcomed for my:

list of ways I can make him think something is his fault;
list of new features he shall feel obliged to subsequently implement;

Actually I think it was quite an intelligent default rule on the part of his firewall to keep me away from his TestDriven.Net source repository while he sleeps, given my "immense" (cough) knowledge of SVN...

In other news, Scott Hanselman released his latest Hanselminutes podcast (about Continuous Integration tools) where he gives a big thumbs up to TestDriven.Net and was kind enough to mention yours truly in there as well in respect of the NCoverExplorer integration - cheers mate! I'm so chuffed I'd like to send the audio file to Mum & Dad too - but there's ever such a slight chance they might not understand it...

NCoverExplorer... Downloads

2006-01-31T14:34:00.000Z

For anyone who (for some insane reason) does not want the combined installation with TestDriven.Net - I can now offer an alternative. Note that this is exactly the same NCoverExplorer version as deployed with TestDriven.Net.

Download the latest NCoverExplorer versions from here.

I have also put together an FAQ which is bundled in the zip file...

Last but by no means least I see that both Scott Hanselman and Peter Waldschmidt had some nice things to say about NCoverExplorer - cheers guys, makes that 3am effort on Saturday night trying unsuccesfully to get through Jamie's firewall to his SVN server seem more worthwhile now... ;-)

NCoverExplorer... First Availability

2006-01-31T09:01:00.000Z

Jamie Cansdale has just announced the availability of his latest build of TestDriven.Net in this posting. As part of the installation it includes NCoverExplorer - you can get the download from here. For those of you lucky enough to have .Net 2.0 installed and using the latest NCover 1.5.x you can start using the "Test with... Coverage" functionality right away - that's a pretty awesome combination...

Sadly at my "day job" they aren't willing to deploy .Net 2.0 to the desktop as yet. That limits us to NCover 1.33 which isn't currently* supported by this TestDriven.Net feature. So we still use NCoverExplorer in "standalone" mode... certainly workable thanks to NAnt/NAntRunner but clearly not as slick!

[Update] I have just played with a new build Jamie is working on which does work with NCover 1.33 - champion mate! No doubt he will announce when it's ready for prime time...

As you will see mentioned in Jamie's posting I'm working on making the source etc available as part of the NCover project, thanks to Peter for the offer! I've also had a kind offer for a download mirror site - will announce here when everything falls into place...

NCoverExplorer... The Debut

2006-01-28T15:23:00.000Z

A little something I knocked together one weekend around a month ago, inspired by the NCoverBrowser tool by Jeff Key. The source code for Jeff's tool was unavailable mainly due to it's usage of a commercial control, so I rewrote it from scratch using the ICSharpCode TextEditor control instead.

What does it do? Once you have used NCover to produce an xml result of your coverage, you can open it in this tool to browse the source code with the coverage information highlighted. I tend to do this as part of a NAnt/NUnit task combination that automatically opens the results for me on a second screen. I find it much more useful having the coverage output side-by-side with the IDE while I add/modify the tests to increase coverage.

Why bother? I wanted to make some tweaks, such as:

Performance (2 secs versus 4 mins for a 12MB file!);
Remembering form state (size and position);
Recent file menu;
Improved highlighting for partial test coverage runs;
Optional nesting of the get_ and set_ properties;
Optional hiding of the visit count pane;
Support any future NCover schema changes;
Create an open source project;

I happened to mention it to Jamie Cansdale, author of the essential TestDriven.Net tool and it is now being integrated into his tool as part of the "Test with... Coverage" functionality he has added recently. In addition Jamie has added a few tweaks of his own such as the VSTS block highlighting style you can see in the screenshot above.

Where do I get it? Good question - our next step is to figure out the hosting and distribution. A few ideas have been thrown in the pot and hopefully we can announce that shortly.

[Update] Download the latest NCoverExplorer versions from here..

Any suggestions for other features you would like to see are welcomed...