Tuesday, April 04, 2006

NCoverExplorer... v1.3.3

Another stack of new features in the latest generation of NCoverExplorer. I've added summary html/xml reporting, filtering capabilities, basic code metrics and very significantly a console version for automated build usage.

Html reporting as I posted about here has been requested by a number of users, including myself! For this release I have implemented summary reports since from the feedback I received they were of most interest. From either the GUI or the the console application you can now generate one of three reports:
  • Module Summary
  • Namespace Summary
  • Module and Namespace Summary
As you can see from this dialog you have a choice of output either directly to html or to xml (the latter having an associated stylesheet for rendering html).


An example of the module summary report is shown here. Note the inclusion of some basic statistics at the top right. You can also see that our "satisfactory coverage threshold" is displayed at the top (90%) and that any graph bars exceeding this threshold (but not 100%) have the uncovered code shown in yellow. So a glance at the report for the red bars will tell you which modules or namespaces are possibly in need of investigation.

Statistics are also available within the NCoverExplorer GUI. Note that these are derived from the NCover coverage report, not the source code. The totals deliberately ignore any "excluded" code.


Filtering offers a way of simplifying down the coverage tree without affecting the coverage totals (unlike "excluding" which recalculates the totals). There are two filters available - hiding all 100% covered nodes (for when you want to focus on the todo list), and hiding all 0%/unvisited nodes (for when unit testing isolated functionality).


My thanks to Lutz Roeder for the public donation of his excellent CommandBar code. You can see the impact in this in the new look to the menus with images and the new Toolbar shown below.


Including nodes that were marked as "excluded" is also now possible. You can think of this as an "undo" feature to override either manual exclusions or those defined in the options dialog. Including a node will also recalculate the related coverage statistics.


NCoverExplorer.Console came about from two features on my list. One of these from Matt Hargett of imeem.com was to enable failing a build if coverage does not meet a particular threshold.

The second item was the ability to produce xml and html coverage summary reports in an automated fashion. As anyone who has used NCover on large projects will testify, coverage.xml files can be very large. Rendering these using xsl while being an elegant and flexible solution is just way too slow, particularly when the coverage file is many megabytes. Merging the coverage.xml file(s) into your CruiseControl.Net buildlog.xml files also slows down the rendering of every single page for that build on the CCNet web server.

My solution to this is based on the assumption that I do not want to see line level coverage information on the build server. I just want to see a summary of the project, modules and perhaps down to namespace level of coverage. By adding NCoverExplorer.Console.exe to the build pipeline we can transform the NCover results very quickly into a tiny summary xml file. With the appropriate stylesheets we can now render coverage summary reports in CCNet in a fraction of the time previously taken.


All of the options available in the GUI in terms of exclusions, thresholds, project name, output as xml and/or html are available.

I have also developed an additional stylesheet for use on the Build Report page of CCNet as shown below:

Since these are just stylesheets transforming the NCoverExplorer xml file they can be customised to your own tastes if required.

The final piece of the puzzle is to automate through NAnt and MSBuild the creation of the xml/html reports. You can of course just use an <exec> task, however I have put together some custom tasks for both tools which offer a bit more flexibility. These are available for download in the NCoverExplorer.Extras zip file at the bottom of this post along with some included examples and the necessary stylesheets for CruiseControl.Net.


If anyone requires it I may put up a blog entry walking through such configuration or perhaps see about getting it documented on the CruiseControl.Net wiki. It is fairly straightforward though:

  1. Copy the NCoverExplorer.xsl and NCoverExplorerSummary.xsl stylsheets from the NCoverExplorer.Extras zip file to your CruiseControl.Net\webdashboard\xsl folder
  2. Edit your CruiseControl.Net\webdashboard\dashboard.config file to point to these stylesheets. I have included an example dashboard.config in the .zip
  3. Modify your NAnt or MSBuild script. As stated you can either execute the console application directly with command line arguments or use my custom tasks. For instance to use the custom NAnt task copy NAnt.NCoverExplorer.Tasks.dll to your NAnt\bin\Tasks\net folder and follow the example syntax contained within "example.build".
  4. Modify your CruiseControl.Net project file (ccnet.config) to use the merge publisher to include the xml output from NCoverExplorer.

For my own sanity, other than any bug-fixes I am intending to put a hold on further development for a little while. This has consumed every evening and weekend for the last couple of months so it's time for a break. Any donations could of course help rekindle my interest ;-)

As always I welcome any feedback. Enjoy!

Download TestDriven.Net 2.0.1578 combined install

Download the latest NCoverExplorer versions from here.

Release Notes
FAQ

19 Comments:

At April 05, 2006 2:32 am, Anonymous Anonymous said...

Nice. Just plain nice.

 
At April 05, 2006 6:54 am, Anonymous Anonymous said...

About dialog needs updating :)

 
At April 07, 2006 7:50 pm, Anonymous Anonymous said...

I just start having a strange problem with NCoverExploer 1.3.3 and 1.3.4.
The NCover xml file is full of things, but opening the report yields nothing in NCE. Everything is greyed out, with statistics of 0%... even though the stats are in the ncover xml file. Restarting Studio didn't make a difference.

Any ideas?

 
At April 07, 2006 8:54 pm, Blogger kiwidude said...

Not something I've seen other than if the code you are executing has been excluded by default (for instance if your assembly ends in ".Tests" is a default exclusion). You would see that code under the "Excluded" node. Excluded code is not included in NCoverExplorer coverage statistics - so if your test method didn't call any non-excluded code the total coverage would be zero!

When you say the coverage.xml file is "full of things" - does it actually have any 'visitcount' values on seqpnt nodes which are not zero?

How are you generating the coverage file - with TestDriven.Net or command line? Can you send me the coverage file?

Please send me an e-mail with the answers above and I can take a look - my e-mail address is on the Help menu (trying to avoid spam).

 
At April 08, 2006 12:50 am, Blogger kiwidude said...

Re the about dialog - I've just pushed another release (1.3.3.1579) up onto this site with a bug fix and an updated About dialog in terms of it's credits.

Is it something else that needs updating about it or did that hit the mark? Or was it the issue with how it looks on Win2K (alpha channel issues)?

 
At April 10, 2006 1:42 pm, Anonymous Anonymous said...

Right. My Coverage.xml file is full of visit counts, and all of the other data one would expect.

But here's the thing. The last TDD.NEt package has changes something. My Coverage.xml files used to be placed in the project directory. They are now getting stuffed in my %profile%/Applicaiton Data/Mutant.../4ProjectName" folder.

Manually loading any files from this location yields no NCover results, even though the file contains the proper data.

Loading and files outside of that area yields exploere data, but those are probably old files. Even if I move one from the new location out to C:\, it doesn't work.

I'm using the latest NCover 1.5 betas for .NET 2.0. Something is definately squirrely.

 
At April 10, 2006 2:02 pm, Anonymous Anonymous said...

Re: missing coverage...

I really don't think this is your problem. Sorry about that. Something about the last two TestDriven.NET dists or their interaction with NCover has totally borked testing for me.

I have two assemblies, A, and ATests. It used to work just fine when I'd profile ATests, it would stats out how much of A I had tested.

Now with the lest versions of everything, profiling ATests yields me a Coverage.xml file with not stat's whatsoever for A; quite totally defeating the purpose. :-/

 
At April 11, 2006 5:38 pm, Blogger kiwidude said...

For anyone wondering what the answer was to the 'missing coverage' issue - it came down to there being no pdb files available for the assemblies being profiled as the user had placed them in the GAC at some point (testing their installer). Love it when it's not my doing... ;-)

 
At May 24, 2006 7:18 pm, Anonymous Anonymous said...

Great piece of software! Thanks!

 
At May 24, 2006 7:19 pm, Anonymous Anonymous said...

Great software! Thanks!

 
At June 26, 2006 9:06 am, Anonymous Anonymous said...

Very nice! I'll be using the reports immediately! =)

One small request: Could you make the NCover merging facility available via the console program and MSBuild task? Thanks!!

 
At June 26, 2006 9:40 pm, Blogger kiwidude said...

Re the merge request - sounds a good idea. Will see if I can sneak it into the 1.3.4 release due in the next week or so...

 
At June 29, 2006 1:46 am, Anonymous Anonymous said...

When I merge two particular coverage files in one order everything is fine. If I merge them in reverse order I get an error saying "Different source code between coverage files means these cannot be merged."

The NCover forums suggest there may be a bug in NCover which causes it to sometimes not include certain nodes. But it seems that NCoverExplorer is able to tolerate this situation at least if I merge the files in the right order.

Since it seems I can still get meaningful reports in spite of the problem, could we add a switch to NCoverExplorer.Console to disable the source consistency check?

If you'd like the coverage files to experiment with please email me at jeff at ingenio dot com.

Thanks!
P.S. Glad to see the command-line merging feature going into v1.3.4.

 
At July 08, 2006 1:18 pm, Blogger kiwidude said...

The merging issue is fixed in the NCoverExplorer 1.3.4 release, as blogged about here

 
At April 05, 2008 6:57 pm, Anonymous Anonymous said...

Hi,

I am using cruisecontrol.net for continuous integration for my project. I want to integrate NCoverExplorer also in it.

In my project I have created separate project for all unit tests and and separate project for the code to be unittested.

How should I change my project.build file to integrate NCoverexplorer in it?

Any help?

Regards.

 
At July 01, 2009 4:54 pm, Anonymous Anonymous said...

Hi,

I am struggling to setup this thing with CCNET. I have the ncoverexplorer report created in xml but with exec. I followed the steps at the dashboard level puting xls at the right place and modifying the dashboard.config but this did not change anything on the dashboard itself.

Is there something I am missing?

Thanks

 
At July 01, 2009 6:35 pm, Blogger kiwidude said...

Anonymous

Re "What are you missing" - did you remember to merge the xml output from the exec into your build xml? In CC.Net have a look at the raw build xml and make sure that the summarised (not raw) NCover output is included.

 
At July 10, 2009 11:26 am, Anonymous Anonymous said...

You were right now, I got it work this way.

Now I am struggling to change the acceptance treshold from 95 to 80. I used /m:80 but it did not change the visual in the dashboard.

/m:80 with /f worked but i don'ty want the build to be broken if the coverage is under the threshold.

Am I missing something obvious again?

Thanks

 
At July 10, 2009 12:13 pm, Blogger kiwidude said...

I suggest that you (and anyone else) post to the forums at NCover.com rather than here. They have forums for legacy NCover and may be able to help. As it is several years since I did any development of it they may be able to give you a more timely and useful response.

 

Post a Comment

<< Home