31 May 2006

TFS Team Build SolutionRoot property

Ok, so you used the New Team Build Type wizard to create your Team Build, but now you find that your code isn’t building because some of the paths are wrong. This is probably due to having hard-coded paths in your project for things like Strong-name keys or, in my case, CodeSmith property sets that have explicit paths to their associated CodeSmith templates.

Need: I need to have the build dump the source files to a specific location on the Team build Server other than the one used by default to accommodate hard-coded paths. This need will exist until a strategy can be defined to move to relative or UNC paths for templates and strong-name keys.

Problem: The New Team Build Wizard doesn’t give you an input element to define the SolutionRoot property in the build.

TeamBuildWizLocationDialog

The SolutionRoot is the folder where the Team Build will store all of the source code files for compilation. This is the property that needs to be overridden in our case.

Issue: By default, the Microsoft.TeamFoundation.Build.targets files defines the SolutionRoot property as:

SolutionRoot = $(BuildDirectoryPath)\$(TeamProject)\$(TeamBuild)\BuildType\..\Sources

So if we have creating a new Build Type under a team project named Argos and setting the New Build Type Wizard’s values to the following:

Team Build Name = NightlyBuild BuildPath = C:\BuildTemp

our resulting final SolutionRoot property would be C:\BuildTemp\Argos\NightlyBuild\Sources. This is not exactly where I would have expected it to go, and not at all where I need it to go.

Solution: To set a new starting path for the Get from source control, you need to override the default SolutionRoot property that team Foundation gives you. This can be done by adding the following snippet to your TFSBuild.proj file for your Team Build.

<SolutionRoot>[Wherever you need source control to start dropping files on Get]</SolutionRoot>

Pretty simple, huh… To make sure that you are overriding the standard behavior, you need to be sure that the SolutionRoot entry that you add comes after the following Import statement:

<Import Project="$(MSBuildExtensionsPath)\Microsoft\VisualStudio\v8.0\TeamBuild\Microsoft.TeamFoundation.Build.targets" />

Because MSBuild works on the premise of “Last one defined wins”, we need to define our property after theirs to make ours stick.

VERY IMPORTANT: The path supplied in the SolutionRoot property is passed to the Clean target. This target is responsible for “cleaning”, (i.e. deleting) the folders that will be used for the build as well as all subfolders. If you set the SolutionRoot to a folder that has critical files below it (like C:\ or C:\Windows), MSBuild WILL TRY TO DELETE IT! I learned this the hard way as I was figuring out all of the above. I hosed a machine so badly that it had to be reimaged. For more info on how TFS Team Build handled this, see my previous post on TFS Team Build Can Run Forever as well as Mike Ruminer’s MSBuild clean and c:\

Conclusion: There are times that you need to force Team Build to utilize a consistent, well know folder structure to make a build work correctly with regard to referenced dependencies. The out-of-the-box behavior is to create a source code structure under the BuildPath. If you need to force it to utilize your standard structure, you can override the SolutionRoot property. The standard behavior of the Team Build is to Clean the SolutionRoot folder before performing a source control Get, so don’t put critical folders in the SolutionRoot property. Make sure that you place your SolutionRoot override after the Import statement to make sure that it “sticks”.

VS 2005 IDE Caching of Workspaces can cause a Team Build to Fail

When you initiate a Team Build from your local client machine, the VS 2005 IDE and Team Explorer will cache the workspace settings for that build on your machine. If at some later point, you delete that Team Build and create a new one that uses the same folder path for it's workspace, you may encounter the following error:

Target InitializeWorkspace: DeleteWorkspaceTask Name="[YourWorkspaceName]" TeamFoundationServerUrl="http://yourTFSServer:8080" Workspace [YourWorkspaceName] does not exist. CreateWorkspaceTask Name="[YourWorkspaceName]" TeamFoundationServerUrl="http://yourTFSServer:8080" MappingFile="WorkspaceMapping.xml" LocalPath="C:\Projects" TeamProject="[yourProject]" C:\Program Files\MSBuild\Microsoft\VisualStudio\v8.0\TeamBuild\Microsoft.TeamFoundation.Build.targets(306,5): error : The path C:\Projects is already mapped in workspace [YourWorkspaceName].
This is due to this caching on your local machine. If you have cleared the workspace from the TFS server, but still see this error, run the following command on your client machine and possibly on the Build Server: tf workspaces /o:* /s:http://yourTFSServer:8080 This will update your local cache with the correct data from the TFS Server. Update the yourTFSServer with your Production TFS server name. Update (22-Jun-2006): After actually re-reading the tf.exe command line documentation for the workspaces command, I noticed that you could also have used the /remove option. this option clears the cache on the local PC for the given server, so the next time you run, the workspace cache is updated from the server. Much better! Here's the command line: tf workspaces /remove:* /server:http://yourTFSServer:8080 More info on the workspaces command can be found on MSDN at http://msdn2.microsoft.com/en-us/library/54dkh0y3(d=ide).aspx

30 May 2006

VSTS Team Builds: Getting diagnostic information into the log

To get the MSBuild system to give you all kids of diagnostic information when it runs a Team Build, open up the TFSBuild.rsp file.  This file contains optional command line arguments that get passed to MSBuild.  To crank up the output, add /v:diag as shown below and then check it back into source control.  On your next run, you will see more information than you will know what to do with!

# This is a response file for MSBuild
# Add custom MSBuild command line options in this file
/v:diag

You can see more items that can be placed into the TFSBuild.rsp file by checking out the MSBuild Command Line Reference at MSDN2

26 May 2006

MSBuild Task, External programs and embedded quotes in the command-line

All right, here’s today’s topic…”How to waste time trying to use the MSBuild <Exec> task to run an external program whose command-line contains embedded quotes”.

Need: I was trying to call to a custom console application that performs code-generation for the passed project. Due to the application needing fully-qualified file paths, I needed to place quotes around the parameters. Here are the relevant snippets from the TFS Team Build .proj file:

<CodeGenUtilityPath>"C:\Program Files\CodeGenerationUtility\CodeGenerationUtility\CodeGenerationUtility.exe"</CodeGenUtilityPath>

<Exec Condition=" '@(SolutionToBuild->'%(HasCodeGeneration)')' == 'true' " Command='$(CodeGenUtilityPath) /f="%(SolutionToBuild.RootDir)%(SolutionToBuild.Directory)" /w="$(WorkspaceName)" /l="$(PathToLog)" ' />

Problem: When run, the following error was encountered:

Task "Exec"

Command: "C:\Program Files\CodeGenerationUtility\CodeGenerationUtility.exe" /f="C:\Projects\Common\Contracts_2_0\ " /w="VPC-XXXX_XXXX_Common SnapBuild" /cl="Common SnapBuild_20060525.7" /r /l="\\Test4\filedrop\Common SnapBuild_20060525.7\CodeGenLog.xml"

CodeGenerationUtility =========================

Given a path to a folder, retrieves the list of files that match the pattern *Codegen?.*, checks them out of VSS, generates code from the .xml files and checks them all back in. – – – <Rest of help text from CodeGenerationUtility.exe removed /> – – –

'/f' is not recognized as an internal or external command, operable program or batch file.

'\\Test4\filedrop\Common' is not recognized as an internal or external command, operable program or batch file.

'" ' is not recognized as an internal or external command,operable program or batch file.

Issue: As you can see, the command parser is correctly interpreting the quotes around the executable path (as shown by the console app’s help being displayed), but it is not passing the app’s command-line args to it. It is instead trying to run these as commands. It is also splitting the command line arguments without regard to the passed quotes around the paths. It sees the space within \\Test4\filedrop\Common SnapBuild_20060525.7\CodeGen.xml as an argument separator. This causes cmd.exe to believe that these are 2 separate arguments/commands. Ok, so how do we deal with this?

First off, let’s have a primer on the <Exec> task (from MSDN):

This task is useful when a specific MSBuild task for the job that you want to perform is not available. One disadvantage of using the Exec task rather than a more specific task is that it cannot gather output from the tool or command that it runs.

The Exec task calls cmd.exe instead of directly invoking a process.

The last line is what concerns us. When you shell out to cmd.exe, you have a new set of rules to follow with regard to quotes around strings in your command-line.

Here is the relevant parts of the help text from running cmd.exe /?:

Starts a new instance of the Windows XP command interpreter

CMD [/A /U] [/Q] [/D] [/E:ON /E:OFF] [/F:ON /F:OFF] [/V:ON /V:OFF] [[/S] [/C /K] string]

/C Carries out the command specified by string and then terminates /K Carries out the command specified by string but remains

If /C or /K is specified, then the remainder of the command line after the switch is processed as a command line, where the following logic is used to process quote (") characters:

1. If all of the following conditions are met, then quote characters on the command line are preserved: - no /S switch - exactly two quote characters - no special characters between the two quote characters, where special is one of: &<>()@^ - there are one or more whitespace characters between the the two quote characters - the string between the two quote characters is the name of an executable file.

2. Otherwise, old behavior is to see if the first character is a quote character and if so, strip the leading character and remove the last quote character on the command line, preserving any text after the last quote character.

So it looks to me like the <Exec> task is calling cmd.exe with the /c option and thus we are seeing the behavior listed in #2 above. This gives us a totally messed up command line to process.

So how do we fix it????

Solution: We follow the old adage “If at first you don’t succeed, give up!” Actually we give up on forcing the MSBuild project to trigger the external app through the <Exec> task and we build our own <fanfare volume=’loud’>Custom Task!</fanfare> to fire this thing off. There a number of benefits to using a custom task for this, including:

  1. We have finer grain control over the running process and can send output to the build log. The <Exec> task does not have this feature.
  2. We can be more descriptive in our project file as to what we are passing. So instead of seeing /f=”C:\Projects\Contracts” /l=”\\SomeServer\filedrop\CodeGenLog.xml” we can now see <CodeGeneration> <SourceFolder>C:\Projects\Contracts</SourceFolder> <LogFilePath>\\SomeServer\filedrop\CodeGenLog.xml</LogFilePath> </CodeGeneration>
  3. We can pass data back to the MSBuild engine upon task completion if we want to.

These are really great reasons to build your own custom task. Another reason is that it is really simple to do. There are a number of blogs and sites out there that describe the process. Instead of adding it to this post, I’m giving you a link to steps I used from Bart De Smet’s blog entry on Custom Build Tasks.

Conclusion: The <Exec> task is great of you need to run a built-in command, such as dir, copy, or even cacls, but it isn’t robust enough to handle a large set of command-line arguments with embedded spaces. If you don’t have the embedded spaces, then you should be fine using <Exec>. If you need a richer method of running an external process, offering logging, output parameters and descriptive metadata entries; then a custom task is the way to go.

24 May 2006

HowTo: Configure TFS Version Control to not allow multiple check outs

This option is set on a “per Team Project” basis, so it allows flexibility for each team to decide how they want to work (unless Management decides for them…)

  1. In Team Explorer, right-click on your Team Project
  2. Select Team Project Settings | Source Control…
  3. On the Check-out settings tab, uncheck the Enable multiple check out box

TFSVC API - Workspace.Get() and GetOptions Enum

I'm writing a console app to do code generation during our Continuous Integration process. To do a Get from TFS Version Control, I'm hitting its managed API, specifically the Workspace object. The Workspace.Get() method has a number of overloads. The one I'm concerned with takes a local or server path and VersionSpec wrapped in a GetRequest object and a GetOptions parameter. I want to force a get in all cases as well as ensure that I will overwrite any existing files on the local file system. Normally I would expect there to be a GetOptions entry for GetAll and Overwrite combined. There isn't, but each of these values does exist on its own within the enumeration. What I did notice is that the values in the enumeration look a lot like bitmask values. GetOptions Enumeration

NameValue
None0
Overwrite1
GetAll2
Preview4
So you can ue bitwise operations to combine these values for use in the Get() operation. The code snippet below will perform the Get() which forces the TFS client to get the file every time and also to overwrite any writable version in the local folder: Dim request as GetRequest request=New GetRequest({local or server path}, VersionSpec.Latest) wkspace.Get(request, _ CType(GetOptions.GetAll + GetOptions.Overwrite, GetOptions)) Since the GetOptions enum is a bitmask you can use the approach above or use a Bitwise-OR to combine these values. wkspace.Get(request, CType(GetOptions.GetAll OR GetOptions.Overwrite, GetOptions))))

How to install TFS Build Server on build machine so it uses a Service Account

I wanted to create a remote build server to allow for Continuous Integration in my project. The TFS Build service needed to run under a "Service Account" (a non-interactive account with limited privileges). To install and configure the machine I followed the following process:

  1. Sign on to the machine with an account with Admin privileges
  2. Give "Service Account" membership in the local administrators group
  3. Log off and login into the server with the "Service Account"
  4. Install the TFS Build Service - which will provide the appropriate and limited permissions to the "Service Account"
  5. Log out
  6. Log back in with an account with Admin privileges and remove the "Service Account" from the administrators group.

This process has been working fine for me for the past 2 weeks

TFS Team Build can run forever

Problem: It is possible for TFS Server to lose track of a remote team build and believe that it is still running even if the build machine has not raised an event back to the TFS Server in a very long time (>36 hrs). It seems that the default timeout for a build is for the TFS server to wait forever. Scenario: I was testing a team build that went awry. My build script deleted a large portion of the C:\ folder and hosed the machine (stop laughing). To stop the build (prior to learning that there is a TFSBUILD STOP command), I shutdown the build server. The machine was hosed and needed to be re-imaged. About 36 hrs later, I reviewed "All Builds" within Team Explorer and noticed that the TFS Server thought that the build was still running. So even after 36 hrs+, TFS hadn't failed the build on a timeout. Fix: I had to use the "TFSBuild.exe Stop" command to inform the TFS server that the build should be aborted. You should also run tf workspaces owner:* server:[MYSERVER] on the Build Server and the Client machine that initiated the build to update the workspace cache which will clean up any stray workspaces. Comment: If you have a failure of a remote build machine during a build, you need to ensure that the build is cancelled on the TFS server or you may have a workspace collision when a new build is run when the machine is back up as the workspace's local path is considered to be still "in use". this "in use" status comes from the Build Machine or Client's workspace cache being out of sync with the Team Foundation Server's database. the tf workspaces command above will update the locaal cache from the server and clean this up. Mike Ruminer has posted a step-by-step listing of the entire event on his blog.

The opening post

Ok, here's the obligatory opening post that gives background and context to the rest of these rantings. I am a developer working for an insurance company. We are a Microsoft house, so I get to play with all of the "cool" and "new" stuff from them. As this blog's name implies, I often hit walls using their products. Now don't get me wrong, Visual Studio and all of its siblings are my bread and butter. I love working with the tools, I just hate those times when I spend half of a day trying to figure out something that should be relatively easy. The worst part is when I finally figure it out, it seems like it shouldn't have taken that long. I especially feel this way when my Dev lead is breathing down my neck about "deadlines" and "iteration plans". I call that "getting burned". Most of the time this is due to lack of documentation from Microsoft and little published knowledge from the wider user community. The current project, and the one that lead me to start this blog, is a proof of concept to implement Microsoft Team Foundation Server. We want to use all of the features of it, but need to first be able to get a successful build process for our Continuous Integration plan going. This includes automated code generation (check out, generate code, check in, build project, repeat for next project) during the "team private build". You would think that this is an easy thing to do. I did too until I started doing Google searches for information. I was amazed that certain specific TFS API related searches returned 1 or 2 hits. There is only a small amount of information out about TFS from Microsoft specifically geared toward working with the APIs and the SDKs aren't much help either. Since I seem to be what is called an "early adopter" of the TFS technology, I'm going to try to relieve the pain that some of you will potentially go through by posting small "snippets" of information that I read/figure out/am told about Visual Studio 2005 and Team Foundation Server. I will also be pointing to other blogs/sites that I feel offer good information. One of the biggies is Buck Hodges blog and the MSDN VSTS Forums which includes one of the first code samples pertaining to the Version Control API. Since I work daily in VB.Net, most if not all of the code samples will be in VB. So that's it for the moment, more to come...