Monday, July 31, 2006

NCoverExplorer... Help Forums

My thanks to Peter Waldschmidt for adding some NCoverExplorer forums to his NCover website. I had a comment today on my blog from Craig Andera who rightly raised the point that it would be nice for users to have some visibility on issues people have raised, when they get fixed etc.

I use FogBugz for my internal bug tracking and feature requests (thanks to Jamie Cansdale!) which is great for what it does. However it does have a limitation in that it is not publicly visible - so you guys can't tell if you are the 1st or the 50th person to have an issue or make a suggestion. With no real other public channel for communication other than my blog I agree it was long overdue for me to to do something about it.

So - please feel free to make your NCoverExplorer suggestions, tell me what works or doesn't - or indeed by special exemption talk about the All Black supremacy in the Bledisloe Cup over Australia (again) is permissible... (Rugby World Cups are not however).

In return I shall try to also post in there about what's on the horizon and of course continue to try to help those in need with NCoverExplorer (or indeed NCover) where I can as hopefully others will too...

NCoverExplorer Forums

Saturday, July 22, 2006

VS.Net... Cancel A Failed Build

I only recently came across this Visual Studio Hack by Michael Wood to fix one of the single most irritating problems with both Visual Studio 2003 and Visual Studio 2005.

[Rant] How Microsoft continue to screw this up is beyond me - do they ever actually use their own products outside of their single assembly demos showing drag/dropping of connections onto forms? Why on earth when you have a referenced project failing to compile would you want to continue trying to compile everything underneath, ending up with screens of irrelevant errors and a Task List that is completely useless? [/Rant]

This macro is an absolute must have in your IDE... just add to the MyMacros -> EnvironmentEvents module (you may already have the WithEvents declaration):
<system.contextstaticattribute()> _
Public WithEvents BuildEvents As EnvDTE.BuildEvents

Private Sub BuildEvents_OnBuildProjConfigDone( _
ByVal Project As String, ByVal ProjectConfig As String, _
ByVal Platform As String, ByVal SolutionConfig As String, _
ByVal Success As Boolean) _
Handles BuildEvents.OnBuildProjConfigDone

If Success = False Then
'The build failed...cancel any further builds.
DTE.ExecuteCommand("Build.Cancel")
End If

End Sub

NCoverExplorer... Module Thresholds

I received an e-mail from a user trying out the new "per module coverage thresholds" functionality which highlighted some things not clear from my examples in the original post. The example I gave for MSBuild (the same comments apply to NAnt or using a .config file) was this:

<NCoverExplorer ...
     ModuleThresholds="MyAssembly1.dll=30;MyAssembly2.dll=80" />

The three things to note are that:
  • Currently the module name match is case sensitive. I have fixed this in the source code for a future NCoverExplorer release. For now, make sure the names match exactly with those you see in the report.

  • There is no path information deliberately in these module names. The comparison done under the covers by NCoverExplorer is against the NCover names without the paths.

  • NCoverExplorer will fail a build (if using the fail minimum switch) if either the overall coverage or any of the individual module thresholds specified are not met.

My thanks to Max Palmer for the feedback. I have also added to the FAQ.

Filed in:

Sunday, July 16, 2006

NAnt and MSBuild Tasks for NCover

I received an e-mail this week from Jay Flowers asking if I would be interested in helping him put together NCover and NCoverExplorer plugins for CI Factory. The intent is to provide these as an alternative for the CoverageEye package he already includes.

I normally use the NAnt <exec> task for running NCover on our build servers, which does the job fine once you have it setup. There is unfortunately a bit of pain involved in truly dynamic scripts ensuring paths with spaces are handled correctly, building delimited lists of assemblies etc. The NCover command line in combination with NUnit can be a frustrating learning curve, not to mention it has changed between versions so even looking for answers in web blog articles can be confusing when they are out of date or "simplified".

To make this a little more painless for both Jay and anyone else interested I ended up creating NAnt and MSBuild tasks for NCover, supporting NCover 1.3.3 and 1.5.x to date.

I have now included these tasks in my existing NCoverExplorer NAnt/MSBuild task assemblies located in NCoverExplorer.Extras.zip, along with all the source code and example .build/.proj files.

Note: These tasks still require that NCover has it's CoverLib.dll COM registered correctly on your machine. You can always do this as part of your NAnt/MSBuild script prior to using these tasks if not using the default NCover MSI installation.

Here is an example of the NAnt <ncover> task at it's simplest for NCover 1.3.3 - note the use of the version attribute. The <ncover> task will default to NCover 1.5.4 command line syntax if this is not present.

[Update 03 Sep 2006: As of build 1.3.5.1789 the <ncover> tasks are self registering without needing local admin rights. The version attribute is also now redundant - it will be picked up from the NCover.Console.exe assembly if not specified]

<ncover program="Tools\NCover\NCover.Console.exe"
commandLineExe="Tools\NUnit\nunit-console.exe"
commandLineArgs="MyApp.UnitTests.dll"
version="1.3.3" />


A more complex NAnt example using NCover 1.5.4 showing additional NUnit arguments, customised logging and xml filenames, a coverage exclusion attribute (in the root namespace) and specific non-test assemblies to report coverage on:

<ncover program="Tools\NCover\NCover.Console.exe"
commandLineExe="Tools\NUnit\nunit-console.exe"
commandLineArgs="MyApp.UnitTests.dll /xml=MyApp.nunit.xml /labels /nologo"
workingDirectory="${build.output.dir}"
coverageFile="MyApp.ncover.xml"
logLevel="Verbose"
logFile="MyApp.ncover.log"
excludeAttributes="CoverageExcludeAttribute">
<assemblies basedir="${build.output.dir}">
<include name="MyApp*.dll" />
<exclude name="MyApp.UnitTests.dll" />
</assemblies>
</ncover>


Here is a simple MSBuild <NCover> task example:

<UsingTask TaskName="MSBuild.NCoverExplorer.Tasks.NCover"
AssemblyFile="MSBuild.NCoverExplorer.Tasks.dll" />
<NCover ToolPath="Tools\NCover\"
CommandLineExe="Tools\NUnit\nunit-console.exe"
CommandLineArgs="MyApp.UnitTests.dll" />


I did manage to confirm a few bugs/unfinished features in NCover 1.5.4 itself while giving this a test, so where possible I have either worked around or excluded these features until they are one day resolved. FYI, they involved the //q and //h command line arguments, which you should avoid using!

Feel free to leave a comment or drop me an e-mail if you find these tasks useful (or find a bug of course!).

Download NCoverExplorer.Extras.zip (CC.Net/NAnt/MSBuild)

[Update: I'm more than happy to answer questions for people adding these into their builds. However rather than using the blog comments can you please put your query in the NCover or NCoverExplorer forums as I blogged about here.]

[Update 03 Sep 2006: NAnt and MSBuild tasks have been updated and improved. See this thread for the changes listed in build 1.3.5.1789 or the 1.3.5 release notes. Documentation can now be found here for MSBuild and here for NAnt (or in the zip file).]

[Update 07 Sep 2006: Bug-fix update for the NCover tasks where multiple arguments specified in the command line. Build 1.3.5.1797 now available at the above links.]

[Update 01 Oct 2006: Updates to the NCoverExplorer tasks to support a new regular expression feature for coverage exclusions and improved documentation. Build 1.3.5.1861 now available. NOTE you must also upgrade your NCoverExplorer version to the latest 1.3.5 beta.]

Wednesday, July 12, 2006

NCoverExplorer... v1.3.4

It's been a little while since the 1.3.3 release, but NCoverExplorer overload meant I had to take a break to keep some sanity. This release combines some important bugfixes with a small number of new features.

Firstly the new features:
  • On the toolbar of the NCoverExplorer GUI you will find four new buttons for navigating unvisited lines and classes. These have shortcut keys of 'N'/'P' (or forward/back mouse buttons) for the next/previous unvisited line in the current class. Use 'Ctrl+N'/'Ctrl+P' for next/previous unvisited class in this namespace (or Ctrl + forward/back mouse buttons). Thanks Jamie for the mouse button suggestion, it rocks!

  • You can use NCoverExplorer.Console.exe to merge and save your NCover coverage.xml files. Coverage exclusions will be applied prior to the save of the consolidated xml file. Wildcards are supported for the coverage filenames, so for instance you can do this:

    ncoverexplorer.console.exe *.ncover.xml /s:coverage.merged.xml

    If using the NAnt or MSBuild tasks from the latest NCoverExplorer.Extras.zip, you can add a new attribute "mergeFileName" (NAnt) or "MergeFileName" (MSBuild) containing the filename for the results.

  • Module level coverage thresholds are now possible using NCoverExplorer.Console.exe, rather than just a single threshold for your project. This is for people wanting more granular control where specific assemblies are deemed unrealistic to attain the same coverage as your overall goal, without seeing "failure" forever on coverage reports.

    For NAnt, add this section within your <ncoverexplorer> task:

    <ncoverexplorer ...
        <moduleThresholds>
            <
    moduleThreshold moduleName="MyAssembly1.dll" satisfactoryCoverage="30" />
            <
    moduleThreshold moduleName="MyAssembly2.dll" satisfactoryCoverage="80" />
        <
    /moduleThresholds>
    <ncoverexplorer>

    For MSBuild, you can add this attribute to the <NCoverExplorer> task:

    <NCoverExplorer ...
         ModuleThresholds="MyAssembly1.dll=30;MyAssembly2.dll=80" />

    For complete examples refer to "example.build" and "example.proj" within the NCoverExplorer.Extras.zip file. If you do not want to use the NAnt/MSBuild tasks then you can use the command line with the /c option for a config file. The syntax is the same as for the NAnt <ncoverexplorer> task above - see the ConsoleExample.config file in your NCoverExplorer folder.

  • There is a new code coverage report, which drills down to classes per namespace per module. This is my "favourite" report we use on our CruiseControl.Net build servers. To generate this report the magic number is "4" for your report type attribute in the command line, NAnt and MSBuild tasks.
  • Updates to the CruiseControl.Net build summary stylesheet, so that it lists all of the assemblies, their coverage and whether they passed/failed as well as the overall project coverage. Note that this stylesheet now has a dependency on a transparent.gif file being placed in your CruiseControl.Net images folder. See the readme in NCoverExplorer.Extras.zip for details.
  • Added the ability to change source code paths for a loaded coverage file in the NCoverExplorer GUI. Sometimes you will be generating coverage files on another machine (such as a build server) that you want to browse - this feature allows you to override the embedded location to find the source code files in.

In terms of bug fixes, the most important that I blogged about recently is a solution for an issue when merging multiple coverage files. There are also fixes for memory leaks and various other minor GUI issues - see the release notes for full details.

As always, please drop me an e-mail through Help->Send Feedback if you have found any bugs or suggestions.

One final point - no doubt most people reading this have heard about Jamie Cansdale "going commercial" with TestDriven.Net. Jamie has been out there upgrading and supporting this great add-in for many years, for which many of us have enjoyed the benefits of "for free". Having had a small taste of what is involved in the last six months with NCoverExplorer you quickly gain an appreciation of just how much of your "life" supporting and enhancing these tools can consume! I wish him all the best in his opportunity to gain a return for those efforts - his continual innovation with TestDriven.Net continues to help fill the many gaps in the Microsoft "out of the box" tools to make for a far more productive development experience.

Download TestDriven.Net 2.0 combined install

Download the latest NCoverExplorer versions from here.

Release Notes
FAQ

Sunday, July 02, 2006

NCoverExplorer... Merging Coverage Files

Several users have reported issues when merging coverage xml files - the merge is aborted with the following dialog:


In response to this post on the NCover forum I explained why this occurs, without being able to provide an immediate workaround still using the merge feature.

I have just modified the way I process the xml files to workaround this problem and will include this fix in the NCoverExplorer 1.3.4 release which I will push out in the next week or two.

While on the subject of merging - I have also enhanced the NCoverExplorer.Console.exe and the NAnt/MSBuild tasks to now allow the automated saving of the results of merging multiple coverage files.

For anyone who can't wait for the official release please drop me an e-mail (use Help-> Send Feedback in NCoverExplorer) and I will give you a link to the current build. Any early feedback to identify any issues with these features or the others I have added for 1.3.4 would be appreciated.