CCStatistics for CruiseControl.Net 1.1

A few days ago CruiseControl.Net 1.1 was officially released. Thanks to all those responsible - it has become an entrenched daily tool for many of us. I've upgraded a couple of build servers so far - well worth doing for the performance improvements and statistics features alone. This also gave cause for me to knock up a supporting tool called "CCStatistics" - more on this below.

When upgrading there are a few things you need to change in your CC.Net project files that are not all listed in the release notes...

1. Remove any <publishExceptions> elements.
2. Change the format of your <weburl> links to jump to the latest report in CCTray. They should now be in the format below, unlike the query-string approach used previously:
   http://localhost/ccnet/server/local/project/MYPROJECTNAME/ViewLatestBuildReport.aspx
   
3. If you want to use the new statistics functionality you need to add a <statistics /> node to your <publishers> section. Without it when you click on the "View Statistics" page you get a nasty exception being thrown.

Speaking of statistics, if you hadn't guessed already this is one of the more interesting additions for me. In my opinion it is a "0.9" version in terms of readiness for release - there are a number of bugs I found I have added to Jira and the documentation is incomplete or misleading in a couple of instances at the time I write this.

That said it has the potential to be a stonkingly good feature. Who doesn't want to see the most common reasons their build fails (when you have ClearCase performance issues like we do) or see their unit tests count, code coverage etc over time all on a single web page? No more endless drilling down into build log hyperlinks...

To use this feature you add a <statistics> node to your <publishers> section in the project file. I recommend you put this at the end of your publishers (after any <merge> element) so that way you can get additional custom statistics from files merged into the build log. When CC.Net runs the project it will then generate in your artifacts folder:

• report.xml (not statistics.xml as the documentation says). Contains all the statistics for builds to date.
• reports.bmp (which is actually a png - JIRA). A graphic illustrating the "TestCount" statistic across the build log cycle.
• statistics.csv (JIRA). Intended to be an exportable alternative to report.xml - but it has garbage output in it currently due to a few bugs.

When you click on the "View Statistics" link from a build project page you will see the standard statistics on a page like the following:


The report is fairly basic in appearance but that should be easily changed by editing the statistics.xsl file. That is next on my list and once done I will contribute it for anyone interested (or feel free to beat me to it!). Strangely the reports.bmp file is not referenced in this stylesheet by default - so the graph produced is not actually being used yet as far as I can tell.

A minor limitation in the current release is that you cannot "replace" statistics with your own ones without editing the xsl. Not the end of the world but again I've posted a feature request on JIRA on ideas to make that more flexible. You can however add new statistics as mentioned here. So for instance if we want to add the code coverage totals for the build as a result of running NCoverExplorer we can do this:

<statistics>
<statisticList>
<firstMatch name="Coverage" xpath="//coverageReport/project/@coverage" />
</statisticList>
</statistics>

This will result in an additional column being added to the statistics page on the right-hand side. You could of course add other NCoverExplorer statistics as well such as lines of code etc.

There is a catch to all this - statistics will only be generated for future builds, not for your legacy ones. That is where my CCStatistics tool comes in. This will parse all your build log files for a project and produce an updated reports.xml, reports.bmp and statistics.csv file as though you had been running CruiseControl.Net 1.1 "forever".

There is another benefit to this tool I can see. Undoubtedly you will over time decide on additional statistics you would like to measure. Provided they are sourced from data that existed in historical build logs, you can regenerate all your statistics to date with the new data included to give you a much better picture of the progress of your project. If the data isn't present (say you just started merging NCoverExplorer code coverage) then the values will be blank.


It should be pretty trivial to use - drop the executable into your CruiseControl.Net\server folder, click on the Load button to list your CC.Net projects, choose which to recalculate statistics for and away it goes...

I've made all the source code available - perhaps the CruiseControl.Net developers might be interested in adding it to their solution and maintaining it? There is also a compiled executable against CC.Net build 1.1.0.2498 (aka CC.Net release 1.1.1). To use the source code just unzip into the same \project source code folder as the other CC.Net projects and add to ccnet.sln.

This was a very quick hack from a few hours work and has the usual "it works on my machine" proviso. There is no care or attention over internationalization and I have only ever used the 1.x versions of CruiseControl.Net. So use at your own "risk" - although as the only file access it does is reading logs and writing the couple of statistics files I don't think there's much to go badly wrong. The code itself has a few "hacks" so as to work without touching the original CC.Net source code - these may be cleaned up at a later date if the dependent objects are enhanced. Note also that this code results in the same "useless" .csv file - that bug is in CC.Net source and even if I did work around it obviously your future builds would still have incorrect lines being published.

If you notice any problems or have suggestions feel free to let me know. Note that it will backup your previous statistics files with a .old prefix so if you notice a problem after running it once you can revert.

Also thanks to Jamie Cansdale for his help today with a sticky issue I had (the NetReflector component only "works" if the assembly containing the type it is trying to instantiate is loaded in memory - just referencing the .dll containing it is not enough).

Download CCStatistics executable (Depends on CCNet 1.1.0.2498)
Download CCStatistics source code

[Update 14-Oct-06: I've pushed up a new version which includes progress bars and timings. When processing a big directory of large files CCStatistics may become unresponsive - don't panic as it will finish eventually. I have also updated the unittests.xsl stylesheet to fix the still outstanding issue in JIRA of test suite failures not showing up that I blogged about previously.]

[Update 22-Sep-07: Damon Carr has significantly updated this tool to work with the latest CC.Net (1.3) which you can can download here as mentioned in the comments below. Thanks also to Drew Noakes for his previous work for compatibility with CC.Net 1.2. Having "served its purpose" at the time for my needs I did not have the time to maintain it myself but am glad that others have found it useful enough to do so. Well done guys...]

Filed in: CruiseControl.Net