Real Incremental Build - Part 4 – Compare Assemblies using ILDASM

Part 1 – for motivation.
Part 2 – Plan for getting only new/updated files.
Part 3 – Out of the box Incremental build with Team Build.

In last part we succeed getting only new/updated files except binaries which were regenerated also no actual change was made to their code – because some reference in their dependency tree was changed.

We want only real updated assemblies.

Again, the trivial solution doesn’t work:

If you try to compare each assembly with its equivalent in the original build you’ll find that each time you build a project, the assembly is different! (even if no code was changed)

Here comes ILDASM (MSIL Disassembler) to the rescue.

This tool takes and assembly (either DLL or EXE) and can create a text file containing all structure and actual IL code of the file.

If you compare 2 disassembled files which should be identical you’ll probably see the following different lines:

1. Time-date stamp
   This parameter is regenerated each time – so it will be different every time.
   
2. MVID
   This is a generated GUID – Regenerated each time.
   
3. .ver
   Lines with .ver arr declarations of the specific version of referenced assemblies.
   Those lines would be changed if some referenced assemblies has changed their versioning.
   This happens due to the use of * in the AssemblyVersion attribute in AssemblyInfo.cs
   You need to decide whether this is a breaking change from your point of view – or not.
   In my case – we don’t work with Signed DLLs, and I don’t mind what is the specific version. 
   
4. PrivateImplementationDetails
   
   I don’t know for sure what that means, but from my experience it changed without any real change happened.
   
5. WARNING:
   
   Those lines happen to be changed.

So here the actual process:

1. You iterate through all the remaining files.
   
2. If the file’s extension is either .dll or .exe – you run ILDASM on this file, . Those are the parameters I used, but to tell the truth – I haven’t investigate them much:
   
   ildasm.exe  assemblyFileName /OUT=out.txt  /ALL /RAWEH  /SOURCE /LINENUM /CAVERBAL /NOBAR /UTF8 /TYPELIST
   
   Run it second time for the original assembly file too.
   
3. Compare the 2 output files (I just iterate through the lines in both files) and ignore the changes which have been described above. (Time-date stamp, MVID, etc.)
   
   I would jump to the next line which doesn’t contain “WARNING:” in case I meet such a line.
   This is because I have seen cases were only one of the assemblies contained such a line, and then it will break you comparison if you not jump to the next valid line.
   
4. If the files are equal – delete the assembly from the drop location.

The result of this procedure is: Diff package containing only new/updated files

Easy it was, wasn’t it?

Real Incremental Build - Part 3 – Out of the box with Team Build

See Part 1 – for motivation.
See Part 2 – Plan for getting only new/updated files.

Ok, so how do we actually perform our plan?

We will write an application which does the following

1. Run Build with the original source:
   
   See Building a Specific Version with Team Build 2008.
   to put here the bottom line – add the following parameter to the command-line arguments:
   /p:GetVersion=version (where version can be C### – which means – Changeset number ###)
   
   for doing this programmatically, use IBuildServer.GetBuildDefinition(teamProject, name) and call CreateBuildRequest.
   
   The following line sets the command line to get the version you want:
   BuildRequest.CommandLineArguments = "/p:GetVersion=" + OriginalVersionSpec;
   
   Call QueueBuild(BuildRequest) to actually start the build. 
   
2. Wait till the build ends:
   
   You can create a Timer and check the build status each time - QueuedBuild.Status
   You’ll need to refresh the build object – using - QueuedBuild.Refresh() before.
   Save the finish time for later use.
   
3. Get changes only:
   
   This is the actual incremental build (all previous steps were preparations).
   You do this the same way like the build before (with the /p:GetVersion).
   But this time, you add another parameter: IncrementalBuild
   
   The full parameter string will look like this: (see the semicolon separator) 
   
   "/p:IncrementalBuild=true;GetVersion=" + CurrentVersionSpec
   
4. Rebuild:
   
   Queue this build as well, and again wait for it to finish.
   
5. Delete old files:
   
   Iterate through all files recursively in the drop location of the incremental build.
   Delete all files with LastWriteTime older than the finish time of the original build.

So now it seems we have what we wanted.

In the Drop location of the incremental build – only newer files will remain.
All files which weren't changed (or executables which weren’t regenerated due to no code change) – were deleted – leaving new/updated files only.

Or is it?

If you’ll look at the results – you’ll see lots of binaries (DLLs and EXEs) which shouldn’t be there.
No code change was done in their projects.
So why are they here?

Because at least one of their references assemblies has changed…

So we succeed partially:
All file types except assemblies – we have updated files only.
Assemblies – we have all files which had a change somewhere in their dependency tree.

Back to square one?

Solution in part 4...