Building Castles On Quicksand

The projects I am involved in for my "day job" are moving to .NET 2.0 so lately it has been head down converting a huge array of wizards, templates, add-ins and of course the code itself. I also rewrote the core of NCoverExplorer over Christmas as part of a future 1.3.6 release but have had no further time for CC.Net, NCover or various other tools I had been tinkering with contributions for. Once things quieten down a bit at work I will be able to crank things up again.

Moving to .NET 2.0 at work has resulted in various amounts of pain thanks to bugs in the .Net framework, the IDE, the designers and the extensibility library. I've mentioned before I am not a fan of VS.Net 2005 and using it in anger these last few weeks has done nothing but lower my opinion. How some of these bugs or design decisions got through "quality control" is beyond belief. Actually I do have a theory on that but I will save that rant for another day.

The reason for this post however is to blog about my experiences with Sandcastle and generating help documentation for VS.Net 2005.

We have been using NDoc of course for our .NET 1.1 based documentation for VS.Net 2003. Unfortunately as was much documented and debated a while ago, Kevin Downs pulled the plug on this product. Part of the rationale given is Microsoft's announcement of Sandcastle, now up to it's third CTP release (Dec 2006) so I figured it might be mature enough to at least try it on for size.

Wrong, Wrong, Wrong.

If you haven't tried it and have been wondering on the sidelines like me - let me save yourselves hours of pain and frustration by suggesting you continue to wait. It is labelled as CTP but it is closer to the prototype end of the scale than the release candidate one - nowhere near ready for community usage, at least for our requirements.

So what got me so underwhelmed about it?

Well for starters the fact that Microsoft haven't yet included any form of command line or GUI means straight away you are into the world of evaluating and choosing other tools to actually make it work. Yes, I know you can argue choice is a good thing and I don't mean to denegrate in any way the effort people out there have put in to try to make up for this shortcoming by Microsoft. However sometimes you just want only one new moving part to add to your build process.

Next you need to install the VS.Net 2005 SDK to get the right Help2 compiler - it is no longer available as a separate small download like it was for VS.Net 2003 with VSHIK.

Then there is the list of "known issues" - mostly problems with the templates which mean your output is going to be kakked so you need to replace them. So that's usually four separate "things" to be downloaded, installed, configured and documented for your build server before you even try to configure a documentation project - straight away you are aware this is not going to be trivial.

So how do you automate Sandcastle? Well, there is a wiki which indicates at least some of the options contributed by the community - a batch script , an MSBuild script and several GUI options, the most popular being an NDoc lookalike called Sandcastle Help File Builder (SHFB). There are VS.Net add-ins as well although I'm after a tool that works as part of our automated build process - not locking up the IDE for hours thanks.

A dozen-step batch script didn't sound too promising to me, so I tried the MSBuild script by Anders Ljusberg first. Simple to deploy, a few tweaks for paths here and there, ran it and *boom* - first technical issue.

By default Sandcastle wants to connect to a web service on the internet - I believe to generate links to Microsoft content. Well that's just great if you live in the magical fluffy world where proxy servers don't exist, or if they do they take IE settings. Unfortunately where I work we have popup dialogs for authentication - which isn't catered for in any way I know of by the standard <system.net> configuration override trick. The "Start" page in Visual Studio.Net 2005 suffers from the same problem.

I couldn't see an obvious way to turn off that type of online linking using the MSBuild script (perhaps someone can tell me how) so that knocked it out of the equation for now. Incidentally I ran the script on my home PC which has no proxy server and hit a different issue, so it's clearly not the only problem you may face with using it.

Next I moved on to SHFB which does have an option in the GUI to set the external linking to "none". This looked far more promising having both a console and GUI version in NDoc style, some help etc - kudos to Eric Woodruff for his efforts. The price you pay over the MSBuild approach is less flexibility in terms of dynamically assigning assembly lists, version numbers etc since it works off configuration files. No major problem though as it is easy enough to knock up a NAnt or MSBuild task to generate the configuration file dynamically. First priority though is getting the output generated.

And that's where problem #2 came up - performance or rather appalling lack thereof. Generating a .CHM file for our 40 assembly application which used to take an hour under NDoc came in at 5 hours with Sandcastle. WTF? Ok, so leaving it running overnight is going to be the Sandcastle way of doing things. BTW, according to SHFB it took 4 minutes 55 seconds - rather wishful thinking!

There were a few problems with the output so I tweaked some settings (like saying I don't want namespace summaries as I dont want big red warnings everywhere) and ran it for VS.Net 2005 output on just three assemblies. Out popped the help - but still the big red warnings! Double checked my GUI configuration - show warnings are set to false. Hmmmm... Equally bad is that there is a horrid empty blue bar near the top of each page - I think it is expecting some sort of input to go in there but can't cope with it not being present.


So after all the research, downloads, configuration, bugs, trial and error I end up with something that takes hours to render and looks crap. Not quite the result I was after - yes I could try to fix the bugs but that's not what I set out to do. I just need to generate some documentation.

So what to do in the mean-time? I had tried the NDoc2005 project on SourceForge but found it completely unusable with bugs and the developers have clearly lost interest.

I then stumbled across something known as NDoc Alpha which appears to be the last work in progress build of the original NDoc developer Kevin Downs. It all sounds very promising on the web page in terms of support for generics and performance boosts.

I downloaded it and gave it a whirl for a .CHM file,and *boom* - 'access is denied' messages. NDoc Alpha copies your assemblies to a shadow folder so as not to lock the originals. The downside is if those assemblies have the read-only flag set it tries to do a delete from the shadow folder to clean up but doesn't check for the flag first. Now that is a pain - I wanted to build from my ClearCase view release folder which has those attributes set. Thankfully there is an option "UseAssemblyShadowCache" which when set to false disables this feature and hence works around the bug - nice.

Once I hacked that I was able to set NDoc Alpha off in motion again... and *boom* hit a series of big nasty exceptions which stopped documentation being generated. After much fiddling I figured out it was an xml tag that had the wrong attribute that VS.Net does not validate as an error. FYI it was <exception name="..."> instead of <exception cref="...">. In later iterations I also found it erroring where the tag had no attribute at all - i.e. <exception>. It appears that the new CHM documentor in this build of NDoc is far stricter than previous NDoc versions - not a bad thing, just it would be nice to have a more useful error message to figure out what is wrong with the tags.

Incidentally I had discovered that NDoc is spitting out Trace messages as it processes elements - that makes life a bit easier to find out what class it was processing when it failed. Fire up your favourite debug trace viewer such as from SysInternals and you hopefully get a hint as to either the method that failed or at least what once had processed successfully in the class.

Finally with all the error messages cleared NDoc Alpha had a clear runway. Boy is it quick! It took a mere 15 minutes to build the documentation from scratch and generate the .CHM file and a similar amount for the VS.Net 2005 version. Now that's just a complete no-brainer compared to Sandcastle. Sorry guys, you have a long, long way to go.

All that remains now is knocking up a new <ndoc2> NAnt/MSBuild task so I can generate the configuration file on the fly. Shouldn't be a major deal, and I will include the <ndoc2> task I write in NCoverExplorer.Extras.zip when I'm done for anyone interested.

The not inconsiderable negative of NDoc Alpha is that it is an end of line product - no source code is available to fix the bugs or anyone to offer support. It looks like Kevin has the last laugh on this for a while longer - his product is indeed still light years ahead of Sandcastle but the community are not going to be able to advance it further. That's frankly a damn shame as this is undoubtedly the best version of NDoc yet - after all the years of effort to let the source code waste away in the cupboard rather than live on in the community by others is a major downer... forgive the pun.

In the mid-long term then undoubtedly provided Microsoft deliver on the hype then they will be the product of choice - particularly for people not looking to use a commercial offering. For now though Sandcastle is a major thumbs down for me.

BTW - does anyone actually like the new "language filter" that is a part of the VS.Net 2005 help? You only work in one language but have to do half a dozen clicks to just see the C# examples by unselecting everything else first? Then if you get a page which doesn't have the language you are after, you go through another clickfest to see if there was an example in another language instead? If only Microsoft would make it a persisted option - setting a default language with an ordered list of alternatives if none of your defaults exist, how hard is that?

If anyone reading this feels I'm being too harsh or you wish to correct me please feel free to slap me round in the comments. My disclaimer is I'm under pressure to get stuff out the door so my "dive in and fix it" threshold is much lower than normal - using the current versions of Sandcastle and the related tools is not for people in a hurry from my experience.

Filed in: sandcastle ndoc