Bypass Gated Check in from TF Checkin Command Line (TFS 2010)

In Team System 2010, Microsoft added long wanted feature – Gated Check in.

This feature lets you promise you won’t break the build by your check in, by converting the check in into a shelveset – running a build with your changes, and only if the build succeed – it check in the code on behalf of you.

It gives you of course the option to bypass this behavior (if you have the required permission).

My problem started in the fact that during the build process itself – we perform a check in…
So this check in fail with the message (and exited with code 1):

Your check-in has been placed into shelveset Gated_XXXXXXX;Domain\User and submitted for validation by build definition \Project-Name\Build-Definition.

I have looked at MSDN for TF Checkin flag that will replace the GUI option to bypass the gated build:

But as you can see, no new option was added.

Luckily, I have run also the TF checkin /? command:

And as you can see, there are few new options here:

1. shelveset
2. bypass
3. login

Those 3 new options came to support this “Gated Check in” feature.
I was actually looking for the /bypass option – which, what a surprise, bypass this feature.

I hope that this MSDN document will be updated soon.

Upgrade from TFS 2008 to TFS 2010 Beta 2 – long wait (literally)

Yes! We have finally got the TFS 2010 Beta 2!!!

I have restored our production Team Foundation Server Databases into a clean computer.
The setup was very easy – just selecting Upgrade flow, and filling some details.

Then it arrived to the most critical part – upgrading the DBs themselves.

It took almost 6 hours…

but… it finished successfully!

So the proof of concept for upgrading has proofed the concept!

Now of course, we still need to see how to upgrade our customized CMMI process – to have all the new staff Microsoft put in the new one. I guess we will use Allen Clark’s Enabling New Features of Visual Studio Team System 2010 Beta 1 in Upgraded Team Projects

Real Incremental Build - Part 4 – Compare Assemblies using ILDASM

Part 1 – for motivation.
Part 2 – Plan for getting only new/updated files.
Part 3 – Out of the box Incremental build with Team Build.

In last part we succeed getting only new/updated files except binaries which were regenerated also no actual change was made to their code – because some reference in their dependency tree was changed.

We want only real updated assemblies.

Again, the trivial solution doesn’t work:

If you try to compare each assembly with its equivalent in the original build you’ll find that each time you build a project, the assembly is different! (even if no code was changed)

Here comes ILDASM (MSIL Disassembler) to the rescue.

This tool takes and assembly (either DLL or EXE) and can create a text file containing all structure and actual IL code of the file.

If you compare 2 disassembled files which should be identical you’ll probably see the following different lines:

1. Time-date stamp
   This parameter is regenerated each time – so it will be different every time.
   
2. MVID
   This is a generated GUID – Regenerated each time.
   
3. .ver
   Lines with .ver arr declarations of the specific version of referenced assemblies.
   Those lines would be changed if some referenced assemblies has changed their versioning.
   This happens due to the use of * in the AssemblyVersion attribute in AssemblyInfo.cs
   You need to decide whether this is a breaking change from your point of view – or not.
   In my case – we don’t work with Signed DLLs, and I don’t mind what is the specific version. 
   
4. PrivateImplementationDetails
   
   I don’t know for sure what that means, but from my experience it changed without any real change happened.
   
5. WARNING:
   
   Those lines happen to be changed.

So here the actual process:

1. You iterate through all the remaining files.
   
2. If the file’s extension is either .dll or .exe – you run ILDASM on this file, . Those are the parameters I used, but to tell the truth – I haven’t investigate them much:
   
   ildasm.exe  assemblyFileName /OUT=out.txt  /ALL /RAWEH  /SOURCE /LINENUM /CAVERBAL /NOBAR /UTF8 /TYPELIST
   
   Run it second time for the original assembly file too.
   
3. Compare the 2 output files (I just iterate through the lines in both files) and ignore the changes which have been described above. (Time-date stamp, MVID, etc.)
   
   I would jump to the next line which doesn’t contain “WARNING:” in case I meet such a line.
   This is because I have seen cases were only one of the assemblies contained such a line, and then it will break you comparison if you not jump to the next valid line.
   
4. If the files are equal – delete the assembly from the drop location.

The result of this procedure is: Diff package containing only new/updated files

Easy it was, wasn’t it?

Real Incremental Build - Part 3 – Out of the box with Team Build

See Part 1 – for motivation.
See Part 2 – Plan for getting only new/updated files.

Ok, so how do we actually perform our plan?

We will write an application which does the following

1. Run Build with the original source:
   
   See Building a Specific Version with Team Build 2008.
   to put here the bottom line – add the following parameter to the command-line arguments:
   /p:GetVersion=version (where version can be C### – which means – Changeset number ###)
   
   for doing this programmatically, use IBuildServer.GetBuildDefinition(teamProject, name) and call CreateBuildRequest.
   
   The following line sets the command line to get the version you want:
   BuildRequest.CommandLineArguments = "/p:GetVersion=" + OriginalVersionSpec;
   
   Call QueueBuild(BuildRequest) to actually start the build. 
   
2. Wait till the build ends:
   
   You can create a Timer and check the build status each time - QueuedBuild.Status
   You’ll need to refresh the build object – using - QueuedBuild.Refresh() before.
   Save the finish time for later use.
   
3. Get changes only:
   
   This is the actual incremental build (all previous steps were preparations).
   You do this the same way like the build before (with the /p:GetVersion).
   But this time, you add another parameter: IncrementalBuild
   
   The full parameter string will look like this: (see the semicolon separator) 
   
   "/p:IncrementalBuild=true;GetVersion=" + CurrentVersionSpec
   
4. Rebuild:
   
   Queue this build as well, and again wait for it to finish.
   
5. Delete old files:
   
   Iterate through all files recursively in the drop location of the incremental build.
   Delete all files with LastWriteTime older than the finish time of the original build.

So now it seems we have what we wanted.

In the Drop location of the incremental build – only newer files will remain.
All files which weren't changed (or executables which weren’t regenerated due to no code change) – were deleted – leaving new/updated files only.

Or is it?

If you’ll look at the results – you’ll see lots of binaries (DLLs and EXEs) which shouldn’t be there.
No code change was done in their projects.
So why are they here?

Because at least one of their references assemblies has changed…

So we succeed partially:
All file types except assemblies – we have updated files only.
Assemblies – we have all files which had a change somewhere in their dependency tree.

Back to square one?

Solution in part 4...

Real Incremental Build - Part 2 – Getting only new/updated files

see part 1 for motivation.

So how do you prepare a package with only the new and updated files?
(whether those are binaries, images, aspx files, resource files, etc.)

The simplistic answer would be to ask the developer which files he touched, and either take them themselves or their products (e.g. the assembly which was generated from the source file).

However there could be lots of developers involved who changed multiple files each.
Try to keep a trace of all those files…

You can use your source control to find all new and changes files, of course, but that still requires someone to sit and analyze which of them can be taken as is, and which of them generates changes in end products.

The knowledge of what file update ends up in a new version of which dll or exe sits inside the project files. So you must use this knowledge automatically.

Which means… Build. 

But wouldn’t a build gives us the whole package again?
No. at least not if you use Incremental Build.

Remember when you change some code in visual studio after you already complied big solution?
the next build finishes faster.
Why? because it rebuild only those projects that have been changed.

So let’s see how can we use it:

1. Build the original source (which already in use in production).
2. Write down the build finish time.
3. Get only the changes from source control (all changes from last original check in till last fix).
4. Build again.
5. Get only new files (newer then the original’s finish time) – or – delete all older files.

How to do it technically?

See part 3.

Real Incremental Build - Part 1 – Motivation for small deployment packages

You have deployed version x of your compiled application to your production environment.
Some bugs were resolved and you want to deploy a fix.

You have 2 options:

1. Deploy a full updated build of the application.
2. Deploy differential package only.

Now, Option 1 will clearly work.
You run your build, take everything from the drop location, and deploy it.

Pros:

• You're sure that you won't missed anything.
• All files are compatible with each other.

Cons:

• The package will have big size (lots of MB – depends on your application size)

At first, the size of the patch looks unimportant.

But if you take into account the time it will take to upload it to your production environment and distributed it between your hundreds of applications instances – this would be unwise not to consider this factor. Or if you let users to download this patch from your website – I guess you’ll want to minimize its size to save your bandwidth usage.

on the next parts we will go from this globally described scenario into more detailed one – based on Microsoft TFS (Team foundation server) source control & Team Build. We will try to show how to create a delta package contains only the new and updated binaries and files. We will meet some implantation problems – and hopefully overcome them.

See part 2.

Tamar Yifrach – תמר יפרח - First Music Album

My sis, Tamar Yifrach, has created a wonderful Folk/Pop/Jewish/Hebrew music album:

The album name is: “תמיד היה מזה קצת בעולם” or “There was always a bit from that in the world”.

You can listen to some tracks from the disc in Tamar Yifrach on MySpace
And for the Hebrew readers – her name is תמר יפרח.

Hope you’ll enjoy it.

I did.