14 June 2006

TFS Team Build: Problems with calling a custom target before building each individual solution - Part I

Ok, so I’ve been banging my head against this wall for about a week now.  I’ve been trying to run a custom target before each individual solution builds with little success…until now!

I am creating a team build for continuous integration purposes.  During the build I have to run a custom task to perform code generation.  At this point you are probably asking yourself “Why the hell is he having a problem with that?”  Just create a BeforeCoreCompile target in your TFSBuild.proj file, call the custom task there and voila…problem solved! 

The problem is that this is not quite what I need.  What happens here is that the build process proceeds as follows (very simplified):

Get all source code in the workspace
Label all source code in the workspace
Run the BeforeCoreCompile action to run a custom task (codegen) for all of the solutions in the build
Compile all of the solutions in the build
Copy all of the binaries to the “drop folder”
Run tests and report

My problem is that code generation template will sometimes use reflection to extract data from previously compiled assemblies to create web services and proxies.  This means that I can’t do all of the code generation at the same time prior to compilation as there are no compiled assemblies to reflect upon.  What I really want to do is:

Get all source code in the workspace
Label all source code in the workspace
For each solution in the build
    Run a custom task (code generation)
      on the current solution
    Compile the solution
Copy all of the binaries to the “drop folder”
Run tests and report

By now you are still saying “Steve, you’re a freakin’ moron!  Haven’t you ever heard of Google?  You are trying to solve a scenario similar to one that Manish Agarawal blogged about  in early May entitled How to call a custom target after building each individual solution (sln) in Team Build.  RTFM!”  Well, as you can tell by my reference, I have read this post and implemented it as well.  I encountered a number of problems and questions during implementation so I said “Steve”, I call myself Steve.  I said “Steve, our company is paying a shit-load of money for Microsoft Support..why the hell not use it?”  So I called into our Microsoft support rep to give me a hand. 

Now just as an aside, I’ve been working with Visual Basic since VB for DOS (yes it actually existed) (yes I am that old) (stop interrupting) right through all versions including the never popular VB 2 and the version that could have been ignored VB 5.  In all of that time, I have never been so lost that I needed to contact Microsoft.  There has always been a wide community of people who have put up great sites, forum posts, BBSes and the like to help each other out.  But in the case of TFS, that community is still forming, so where do you go for assistance?  Right to the horse’s mouth…Microsoft.

So getting back to my story, I contact our support rep and he hooks me up with a PSS engineer out in Redmond who points me to a fantastic blog entry entitled How to call a custom target after building each individual solution (sln) in Team Build.  DUH!  I let them know that I tried it and was hoping that there would be a better solution than overriding the main target in the Microsoft.TeamFoundation.Build.targets file!  They still ask that I try it out, so being a good little doobie, I comply. 

Here’s the code that I added to my TFSBuild.proj file:

<Target Name="CoreCompile">
    <!-- Call the compile for each configuration -->
    <MSBuild
        Projects="$(SolutionRoot)\TeamBuildTypes\
                     $(BuildType)\tfsbuild.proj
"
        Targets="RunCoreCompileWithConfiguration"
        Properties="SolutionRoot=$(SolutionRoot);
                    Platform=%(ConfigurationToBuild.PlatformToBuild);
                    Flavor=%(ConfigurationToBuild.FlavorToBuild);
" />
  </Target>

  <Target Name="RunCoreCompileWithConfiguration">
    <!-- Call the compile for each project (
         
per configuration), with your custom target -->
    <MSBuild
            Projects="$(SolutionRoot)\TeamBuildTypes\
                      $(BuildType)\tfsbuild.proj
"
            Targets="CommonPreCompileStep;RunCoreCompileForProject"
            RunEachTargetSeparately="true" 
            Properties="Platform=$(Platform);
                        Flavor=$(Flavor);
                        ProjectToBuild=%(SolutionToBuild.Identity)
" />
  </Target>

  <Target Name="CommonPreCompileStep">
    <!-- Custom target to execute after
           building each project/solution
-->
    <Message Text="Common precompile target to 
                   execute after building solution
                   $(ProjectToBuild)
"/>
  </Target>

  <Target Name="RunCoreCompileForProject">
    <!-- Actual compile for a particular
             {configuration, project} pair
-->
    <MSBuild
          Condition="'$(ProjectToBuild)'!='' "
          Projects="$(ProjectToBuild)"
          Properties="Configuration=$(Flavor);
                      Platform=$(Platform);
                      SkipInvalidConfigurations=true;
"
          Targets="ReBuild" />
  </Target>

Lo and behold, the clouds part, the sky turns a wonderful shade of blue, an angelic chorus appears, world hunger is eradicated, and my code generation task is running in lock-step with the compilation task.   All is now well with the world, I can go back to coding on my pet project…but wait, why did my build fail?? 

Let’s open up the build log and see.  It looks like everything was moving along swimmingly when [look of consternation] WTF! the log just stops dead in the middle of the run.   It looks like something went wrong and I have no indication as to the problem because the log is truncated. SHIT!  So what does one do in this case?  We turn on verbose messaging and hope that more info can be gleaned from the extra data in the log.  Run the build again and it fails as expected.  Check the log and – WHAMMO –  this log is truncated as well.  DOUBLE SHIT!   What to do, what to do?  I know, use the expensive support folks at Microsoft! 

So I get back in touch with my support folks and we go through a few build, email a few files around and finally the PSS guys asks me to look into the Application Event Log on the build box.  DUH!  Why didn’t I think to look there!  We see an exception from TFSBuild that looks like this:

Event Type: Error
Event Source: TFS Build
Event Category: None
Event ID: 3000
Date:  6/10/2006
Time:  9:12:20 PM
User:  N/A
Computer: MachineName

Description:
TF53010: An unexpected condition has occurred in a Team Foundation component. The information contained here should be made available to your site administrative staff.

Technical Information (for the administrative staff):
Date (UTC): 6/11/2006 1:12:20 AM
Machine: MachineName
Application Domain: msbuild.exe
Assembly: Microsoft.TeamFoundation.Build.Common, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; v2.0.50727
Process Details:
  Process Name: MSBuild
  Process Id: 1520
  Thread Id: 2432
  Account name: Domain\User

Detailed Message: Object reference not set to an instance of an object.
Exception Message: Object reference not set to an instance of an object. (type NullReferenceException)

Exception Stack Trace:  at Microsoft.TeamFoundation.Build.BuildLogger.WriteCompilationMessageHeader()
                                 at Microsoft.TeamFoundation.Build.BuildLogger.LogBuildErrorEvent(Object sender, BuildErrorEventArgs errorEventArg)

It looks like the BuildLogger crapped out in the middle of the run.  The WriteCompilationMessageHeader() method was kind enough to leave us this message when it blew up, but then it rethrew the exception and nobody up the chain caught it, so the logger blew out taking our build with it…NICE!  So why did the BuildLogger die? 

Well it seems that my override of CoreCompile didn’t perform some setup tasks that the logger expected.  There are 2  TeamBuildMessage tasks that I needed to copy from the original to my override CoreCompile.  These are the items that the logger needed but couldn’t find.  The updated targets are below:

<Target Name="CoreCompile">
    <!--
Call the compile for each configuration -->  
    <
MSBuild
        
Projects="$(SolutionRoot)\TeamBuildTypes\
                     $(BuildType)\tfsbuild.proj
"
       
Targets="RunCoreCompileWithConfiguration"
       
Properties="SolutionRoot=$(SolutionRoot);
                    Platform=%(ConfigurationToBuild.PlatformToBuild);
                    Flavor=%(ConfigurationToBuild.FlavorToBuild);
"
/>
</
Target>

 

<Target Name="RunCoreCompileWithConfiguration">
   
<!--
Call the compile for each project
          (per configuration), with your custom target
-->

    <TeamBuildMessage
       
Tag="Configuration"
       
Condition=" '$(IsDesktopBuild)'!='true' "
       
Value="$(Flavor)" />

    <TeamBuildMessage
       
Tag="Platform"
       
Condition=" '$(IsDesktopBuild)'!='true' "
       
Value="$(Platform)" />

    <MSBuild
       
Projects="$(SolutionRoot)\TeamBuildTypes\
                     $(BuildType)\tfsbuild.proj”
       
Targets="CommonPreCompileStep;
                 RunCoreCompileForProject
"
       
RunEachTargetSeparately="true"
       
Properties="Platform=$(Platform);
                    Flavor=$(Flavor);
                    ProjectToBuild=%(SolutionToBuild.Identity);
                    SourceFolder=%(SolutionToBuild.RootDir)%(SolutionToBuild.Directory);
                    
HasCodeGeneration=%(SolutionToBuild.HasCodeGeneration);
                    SolutionName=%(SolutionToBuild.Name);
                    FolderSearchOrder=%(SolutionToBuild.CodeGenerationSearchOrder)
"
/>
</
Target>

<Target Name="CommonPreCompileStep"
       
Condition=" '$(IsDesktopBuild)'!='true' ">
    <!--
Custom target to execute before
           building each project/solution
-->
   
<
Message Importance="high"
            
Text="CommonPreCompileStep->CodeGeneration:
                    $(HasCodeGeneration) ($(ProjectToBuild))
"
/>

    <CodeGenUtility
       
Condition=" '$(HasCodeGeneration)' == 'true' "
       
SolutionName="$(SolutionName)"
       
WorkspaceName="$(WorkspaceName)"
       
LogFilePath="$(PathToLog)"
       
FolderSearchOrder="$(FolderSearchOrder)"
       
Debug="false"
       
IsRecursive="true"
       
ForceUndoCheckOut="false"
       
CheckInLabel="$(BuildNumber)"
       
CodeGenUtilityPath="$(CodeGenUtilityPath)"
       
SourceFolder="$(SourceFolder)" />
</
Target>

<Target Name="RunCoreCompileForProject">
   
<!--
Actual compile for a particular
            {configuration, project} pair
-->
   
<
Message
       
Importance="high"
       
Text="RunCoreCompileForProject->Running: $(ProjectToBuild)"
/>

    <MSBuild
       
Condition="'$(ProjectToBuild)'!='' "
       
Projects="$(ProjectToBuild)"
       
Properties="Configuration=$(Flavor);
                    Platform=$(Platform);
                    SkipInvalidConfigurations=true;
"
       
Targets="ReBuild" />
</
Target>

Still not sure what those 2 TeamBuildMessage tasks are specifically for, but they fixed the logger issue. 

Also, if you take a look at the MSBuild task in the RunCoreCompileWithConfiguration target, you will see that I am now passing a larger set of Properties.  These are the metadata elements that I have added to my SolutionToBuild Items and others that are needed for my custom CodeGenUtility task.  If we didn’t pass these pieces of metadata as properties here, they wouldn’t be available to CommonPreCompileStep and RunCoreCompileForProject.  This is because we are in effect looping through the SolutionToBuild collection and processing each one individually.  We use SolutionToBuild.Identity to pass the solution’s path and the we pull the other pieces of the current SolutionsToBuild’s metadata out as properties as well.  You can add any metadata that you wish to them and pull it out in custom tasks like above so that they get passed around with their corresponding solution.  Here is one of my SolutionToBuild entries:

<SolutionToBuild
   
Include="$(SolutionRoot)\Common\Interfaces\Interfaces.sln"
>
    <
Name>Common.Interfaces</Name>
    <
HasCodeGeneration>false</HasCodeGeneration>
    <
CodeGenerationSearchOrder></CodeGenerationSearchOrder>
</
SolutionToBuild>

Observation:  CoreCompile in Microsoft.TeamFoundation.Build.targets contains 17 different tasks, whereas the ones from Manish’s blog contain just the MSBuild task.  Doesn’t this seem a little odd to you?  I suspect that if the Team Foundation development team put all of these tasks into CoreCompile, they must believe that they are important.  Judging by my logger crash, I believe that they are correct. 

Conclusion:  So now I have my custom task running as expected, the logger is no longer crashing the build, and the angelic chorus has appeared once again to serenade me through my day.  All of my problems are solved…Nope, sorry!  Now it seems that my tests refuse to run… looks like I’m missing some more tasks. 

My trials and tribulations will continue in Part II of this series…

 

4 comments:

Anonymous said...

Hi Steve,

Your solution saved me from wasting more time on this.

I have referenced your solution:
http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=479293&SiteID=1

Thanks again.

Sean

Buck Hodges said...

In TFS 2008, this is very simple to do. See the following short blog post on how to do it.

http://blogs.msdn.com/aaronhallberg/archive/2007/08/07/calling-custom-targets-within-team-build.aspx

Buck

Michael said...

Excellent Post. However, instead of calling a custom target for each solution. Do you know how to call a custom target after each compilation of a project within a solution? Buck Hodges post was great as well but that calls a project and a target within a project, but not a custom target for each project.

Thanks Mike

Steven St Jean said...

I'm not sure I understand the question but that nevers stops me. ;-)

You would probably have to create a Flag property to signify that you want the additional processing during the TeamBuild but at no other time. Then modify the Project file (.csproj or .vbproj) for each project to have an additional Target that gets invoked when the Flag property is True. Better yet, if all the projects need the same post-build step, create a custom target file that the project files Import. You would then set the Flag to True in the SolutionToBuild's Properties element of your TFSBuild.proj file.

Check out Aaron Hallberg's post on how to use the Properties element.
http://blogs.msdn.com/aaronhallberg/archive/2007/03/19/passing-custom-properties-to-individual-solutions-in-team-build.aspx

Post a Comment