Team Build: Using mapped drives gives a read or write error

I recently answered a posting by Paul Fedory in the MSDN Build Automation forum that described a situation in which his projects were using mapped drives for storage of intermediate files and the build was failing under Team Build even though the build worked from the command-line.  This problem looked an awful lot like a problem my team encountered a while back when we started using mapped drives for our references.  I blogged about how we used the mapped drive to help us deal with dependencies in our builds between Team Projects, but I don't think that the posting would come up on a Google search for the mapped drive problem, so I'm going to make this post a little more descriptive.

Paul's Question:

Hi, I'm having some trouble getting my auto-generated Team Build going.  So far I've only changed some of the inherited 'skip' properties to true (ie. skipclean, skipget). 
I get an error when MSBuild runs vcbuild.exe on my VC++ project file. It can't create the directory for the intermediate files.  This directory is stored on another disk (the J drive), which happens to be just a shared folder on my C drive mapped to a network drive (the J drive).  The Team Build Service is logged on as myself, on my computer.  The permissions on this shared intermediate directory are fine (shouldn't the team build service running as me be able to create any directory it wants?)
When I try running this vcbuild.exe command as it is in the build log snippet below, it works fine from the command line.  And the directory is not write-protected.  Is there some issue with accessing drives on the same computer from a service I'm not aware of, or do I need to update some workspace mappings?
Here's a snippet from my build log:
Target Build:
  Target L2CVT32:
    C:\Program Files\Microsoft Visual Studio 8\Common7\IDE\..\..\vc\vcpackages\vcbuild.exe /override:"C:\[BuildPath]\BuildType\TFSBuild.vsprops" "C:\codeintermediate\[BuildPath]\Sources\[Project]\LIB\L2cvt32.vcproj" "Release|Win32"
            L2CVT32 : warning PRJ0009: Build log could not be opened for writing.
            L2CVT32 : error PRJ0007: Could not create output directory 'j:\intermediate\l2cvt32\rel32'.
            L2CVT32 : error PRJ0006: Could not open the temporary file 'j:\intermediate\l2cvt32\rel32\RSP0000013996732.rsp'. Make sure the file exists and that the directory is not write-protected.
            Project : error PRJ0003: Error spawning '/O2'.
        Done building target "L2CVT32" in project "LIB.sln" -- FAILED.

Issue: When utilizing a mapped drive in your build, access errors occur when run under Team Build even if the build service account has full rights to the mapped folder.

Problem:   The problem lies in how the mapping is created.  Drive mappings are only created during "interactive" sessions.

Solution: To have the mapping persist so that the Team Build Service can use it, the SUBST command must be run using the System account on the machine. 

Our Implementation of the Solution: One of our devs built a little command line app that loads itself as a service that runs on startup.  This service runs SUBST.EXE under the System account to make the mapping available to our builds.  We use this little tool on all of our developer and build machines. 

TFS Cool Tool of the Day

I'm still catching up on my blog reading.  Brian Harry has just written about a new add-in from Attrice, the makers of the wonderful "Team Foundation Sidekicks" tools.  The new add-in gives some much needed functionality to the VS IDE. 

They've added "Find items" which behaves like VSS did when you needed to do a search through a source code tree for a specific file name. 

The other tool in this delivery is "Dynamic History".  This godsend-of-a-tool basically does one simple thing that Microsoft forgot.  It updates the history window whenever you click on a file in the Source Control Explorer.  This "tracking" saves you from having to right-click on the new item and then selecting "View History" over and over and over again.

Kudos to all the guys over at Attrice for delivering a productivity booster and sanity-saver all in one!

Attrice: Team Foundation Sidekick add-in

TFS Moves a Little Closer to Being a Good Continuous Integration Platform

I was just catching up on my blog reading when I came across a post from Buck Hodges that shows how to remove one of the biggest pain-points in using Team Build for a Continuous integration process, listing tests to run.

The need to specify the tests that will be run in the CI build by adding them to a test list in the .vsmdi file was a real pain. To make matters worse, you had to either use VSTS for Testers or VSTS Team Suite to edit them, thus incurring extra licensing fees. What should have been done is that if no test list is given in the TFSBuild.proj file, all of the tests in the entire build should have been discovered through reflection and run. This behavior is similar to the way NUnit does it.

Buck's article describes a new tool that he and his team (Team Build) have written that eliminate the need for the metadata files and test lists (.vsmdi). This is a great addition to the CI arsenal under TFS. This functionality will be moved into a Team Foundation Power Tool and Team Build will be updated to support it in a future release.

All I have to say is "Thank's Guys" and it looks like I now owe Buck at least 3 beers. Keep up the good work!

Buck Hodges: How to Run Tests Without Metadata Files and Test Lists (.vsmdi files)

MSBuild Can't Find Secondary References

Hi All and welcome to another installment of "Shouting at the Wall!", I'm your host Steve St.Jean.

On today's show we are going to shout at the wall that is Microsoft's "Decider of things to be fixed."  I have no idea who this person actually is, so I'll blame the whole company.  Here's the story...

Problem: You have an Application "A" that uses Class "Foo" which resides in Assembly "B".  Class "Foo" implements Interface "IFoo" which resides in Assembly "C".  So Application "A" holds a direct reference to Assembly "B" and Assembly "B" holds a direct reference to Assembly "C", but Application "A" does not hold any reference to Assembly "C". 

Click thumbnail for full-sized image 

 When you bring all three of these projects into a Visual Studio 2005 solution and build them, the compilation runs without error, but when you run the same solution through MSBuild, you find that the build fails with an exception stating that the build cannot find a the reference to Assembly "C".

error BC30009: Reference required to assembly "C, Version=1.0.0.0, Culture=neutral, PublicKeyToken=2c72757e0bcb9887' containing the implemented interface 'IFoo'. Add one to your project.

Issue: The cause of this problem is that since Application "A" doesn't have a direct reference to Assembly "C", MSBuild cannot find the Interface "IFoo" that it needs to complete Class "Foo".  What I am seeing is that MSBuild will start resolving references for Application "A"; it finds that Assembly "B" is referenced, so it reads the project file for Assembly "B" and finds where the its binary was compiled.  This binary is copied into Application "A"'s \bin folder and then Application "A" is compiled.  Unfortunately, MSBuild doesn't have the functionality to read Assembly "B"'s project file to find its dependencies, so we get this error.  To get Application "A" to build, you need to add a reference to Assembly "C" even though none of its classes are used directly by the application.

The reason that the Visual Studio 2005 IDE can build this kind of solution is that it can figure out where Assembly "C"'s binary resides prior to compilation of Application "A".  I have spoken with Microsoft Premier Support on this subject.  They have stated that Visual Studio's behavior is a "side-effect" of other processing that the IDE has to do.

I asked Microsoft to fix the problem with MSBuild so that it walks the dependency tree within the Solution to find the dependencies correctly.  The result would be that MSBuild behaves like the Visual Studio IDE.

Ok, now that I've laid out what the problem is, let me give you the response that Microsoft Support gave me when I asked for MSBuild to be fixed.  After they spent some time looking into the issue, I was informed that they would be fixing the problem in a future release.  The "fix" they gave was that they would "modify" the Visual Studio IDE to behave like MSBuild does.  The VS IDE would throw the same error BC30009 in this situation. 

Gripe:  I am not happy with the answer that Microsoft Support passed to me from the dev team.  I don't think that selecting the "easier" fix is necessarily the best way to fix this problem.  My dev leads have stated to me that they don't think that having to add direct references to secondary assemblies is a good idea.  The big problem is that Microsoft build a cool little feature into the VS 2005 IDE labeled "Remove unused references".  After you get your build up and running, all you need to have that build come crashing down around you is to have a conscientious developer that runs this feature. 

I'd prefer to see MSBuild "upgraded" to support looking through the solution's projects to find the references rather than "downgrading"  the VS 2005 IDE.  More than any other reason though, I believe that fixing MSBuild is the "right" thing to do.  So I'd like to give a great big "DUH" to the person or persons at Microsoft that came up with this "fix".  I don't think that this decision is bad enough to get an article on "The Daily WTF", but it's close.

 

I've attached the test solution that I sent to Microsoft Support to illustrate the issue.  If you want to see what I've been complaining about, just extract it and compile it in the IDE, then create a standard Team Build for it.  The VS IDE will build it but the Team Build will fail. 

Attachment:  MSBuildBadRefs.zip