03 December 2006

XSD.exe and MSDataSetGenerator operate differently

Hi all,

Wasn't that title just an attention grabber.  Oh boy, a dissertation on the differences between esoteric code generation tools.  Break out the popcorn and SnoCaps and grab me a soda!

My real reason for writing this entry is not to bore the hell out of you (that's just and added bonus), but rather to hopefully help you not waste large portions of the workday trying to figure out where things are going wrong in a CI build that also does code generation.

Scenario:  When building our CI process, we create two new Build configurations, CodeGenDebug and CodeGenRelease.  These configurations are copies of the standard Debug and Release configurations (respectively) which are used to signal that code generation is required during this build. 

The two configurations behave in exactly the same way except that CodeGenRelease will check in the generated code whereas CodeGenDebug will not.  We use the CodeGenRelease configuration within the CI process and CodeGenDebug is used by the developers to quickly regenerate all of the code in the solution.  This is a real time saver for our teams.

Problem:  Our problem cropped up when one of my dev leads started running the CodeGenDebug build on her machine.  She noticed that when she used the "Run custom tool" menu item on an XSD file, the code was generated correctly and there were no errors shown in the Errors pane.  When she later ran the CodeGenDebug build, a whole plethora of errors were reported by the IDE.

Issue:  What was happening is that when you select "Run custom tool" for an XSD file, the IDE uses the associated MSDatasetGenerator to generate the dataset code.  This code is placed into a file called <XSDFilename>.Designer.vb (or .cs).  Our code generation tools use XSD.exe to generate the dataset.  XSD.exe places the generated code into <XSDFilename>.vb (or .cs).  She had already used "Run custom tool" during her development so when the code generation ran, the project now contained two files that defined the same exact classes.  This is the root of all of those errors.  After doing some local research and speaking with Microsoft support, we found that MSDatasetGenerator and XSD.exe were written by different teams.  Even though they use the same underlying APIs to generate the dataset code, they do it to different files.  Also, there is currently no way to tell XSD.exe what you would like the output file to be named.

Solution:  After conferring with Microsoft Support for a bit, they pointed us to a some Microsoft Connect feedback by the dev team and also came back with a couple of workarounds.  The body of the response is below:


"... So with that said, I see two approaches to this issue:

1) your custom task instantiates an instance of devenv via COM and we use the envdte object model to run the method RunCustomTool off of the VSProjectItem.

2) Instantiating MSDataSetGenerator

a. It might be possible to instantiate the MSDataSetGenerator via COM CLSID ({E76D53CC-3D4F-40A2-BD4D-4F3419755476}) note, this CLSID is under the HKLM\SOFTWARE\Microsoft\VisualStudio\8.0\CLSID\ hive not the usual HKCR\CLSID node.

b. Once we instantiate the COM object, we need to implement IServiceProvider (not the one in the System namespace but the OLE version that has the method QueryService)

c. We queryinterface for IObjectwithSite on the MSDataSetGenerator instance.

d. Call IObjectwithSite::SetSite passing in our object that implements IServiceprovider.  This is necessary because the custom tool uses the IServiceProvider to get information from the host.  For example, the host is usually devenv.exe.  devenv will pass its implementation of IServiceProvider to the custom tool.  The custom tool will call IServiceProvider::QueryService to get SID_SVSMDCodeDomProvider (IID_IVSMDCodeDomProvider) which provides access to the Code DOM provider for C#, VB.NET, J#, etc..  In your scenario, the host is your custom task.  Your custom task will implement IServiceProvider::QueryService.  You’ll pass back whatever language i.e. vbcodeprovider, csharpcodeprovider, etc…

e. The object that is returned in (a), we queryinterface for IVSSingleFileGenerator

f. Call IVsSingleFileGenerator::Generate

I wish (2) was easier above where we just instantiate the generator/custom tool via COM but the above is the minimum we would need to do.  Not all custom tools implement IObjectwithSite so it would be easier to call Generate in that scenario but IServiceProvider also provides access to services like VSProjectItem, IVshierarchy.  That implies that you might need to provide additional services in your queryservice implementation.  VsProjectitem allows the custom tool to get information like name for the particular project item i.e. xsd file."


Both of these workarounds would have required us writing a bunch of COM compatible code  This is not an good solution as far as I'm concerned. 

Since we only really wanted to be sure that the XSD.exe code eventually winds up in <XSDFilename>.Designer.vb and that the the "same class in two files" problem is resolved, we decided to go with a simple solution.  After XSD.exe is run, we will copy the <XSDFilename>.vb file to <XSDFilename>.Designer.vb and then overwrite the <XSDFilename>.vb file with a zero length text file.  This solution will work for us in the short term. 

As a longer term fix, a feature request has been made to Microsoft which allow XSD.exe to take a command-line parameter that will tell it the name of the file to output.  Hopefully, this feature makes the cut and gets in some time soon.

The ORCAS feature list related to Team Build

I just finished reading Buck Hodges' posting about the Team Build features that are forthcoming in Orcas.  It's a laundry list of items that I've been struggling with in the CI space for months and months now. 

"Brian Harry posted a TFS roadmap.  I'd like to expand on the portion that describes the features specific to Team Build.  Keep in mind that these features can change before Orcas ships, so there are no guarantees that it will match what's described below.

Build

  1. Support multi-threaded builds with the new MSBuild.
  2. Continuous Integration – There are many components to this, including build queuing and queue management, drop management (so that users can set policies for when builds should be automatically deleted), and build triggers that allows configuration of exactly how when CI builds should be triggered, for example – every checkin, rolling build (completion of one build starts the next), etc.
  3. Improved ability to specify what source, versions of source, and other build properties.
  4. Improved extensibility of the build targets – such as ability to easily execute targets before and after each solution/project is built.
  5. Improved ability to manage multiple build machines.
  6. Stop and delete builds from within VS.
  7. .NET Object model for programming against the build server.
  8. Simplified ability to specify what tests get run as part of a build.
  9. The ability to story build definitions anywhere in the version control hierarchy."

The most encouraging statement in his post was:

"Continuous Integration is the flagship feature for Team Build in Orcas."

So I'll start keeping abreast of the work being done in Orcas and make sure that my CI and Enterprise builds are compatible, or at least can be easily migrated to the new version.

Thanks for the update Buck and please keep us informed of the cool new stuff you guys are getting ready for us!

- Steve