Making Core internal libraries accessible and maintainable – Part 1

There are always design challenges around long term maintenance of code from start to finish.  It is almost always overlooked and quickly discarded as something “not worth the time” on an already rushed development timeframe for a “business critical bug or feature” that needs to get done.

We all know the following (we = programmers):

  1. Everything, no matter what, will always somehow be critical and necessary and top priority to business groups.
  2. Procrastination of upfront design leads to things similar to the fall of the Roman Empire.

There are infinite issues that can arise from poorly planned and often overlooked items such as: source control structure, solution/project structure and file naming.  Down stream things can get hectic throughout the ALM on its way to production.

To illustrate, here’s a random issue that 99% of the time would never be planned for:

Application A:

Code and Build definitions reside in Team Project 1
Repository Path: $/Team Project 1/Application A/Main/

CoreLib C:

Code and Build definitions reside in Team Project 2
Repository Path: $/Team Project 2/CoreLib C/Main/

The more I delve into the world of source control systems and workspaces I discover that it is an avoided topic amongst most developers.  “Ignorance is bliss and if it works magically: I don’t care” is the typical attitude.  Lets assume a developer adds a project reference into Application A from a csproj in CoreLib C via local file system created by the Get from source control.

The workspace local path essentially doesn’t matter (most cases it shouldn’t matter).  However, it can if its treated as magic.  The project reference ultimately becomes a relative path reference to the other csproj.  If my root $ was mapped to D:\Dev it would look like this:

D:\Dev\Team Project 1\Application A\Main\Code\MyApplication\MyApplication.Business.Entities\MyApplication.Business.Entities.csproj (130 characters)
D:\Dev\Team Project 2\CoreLib C\Main\Code\CoreLib.Domain.Entities\CoreLib.Domain.Entities.csproj (96 characters)

The reference inside MyApplication.Business.Entities.csproj would relative-path to:

../../../../../../Team Project 2/CoreLib C/Main/Code/CoreLib.Domain.Entities/CoreLib.Domain.Entities.csproj (107 characters)

The path lengths don’t seem that long to the developer.  However, the build agent hates it.  When MSBuild (yes – even in 2010 workflows MSBuild is still used to do the actual compilation) evaluates a project for dependencies to determine it’s build order, it combines the two full paths.

Lets state the build server is configured to make its workspace paths off of (a very typical setting): D:\Builds\$(BuildDefinitionPath)

When it starts getting sources it will create its working directory the build definition path variable becomes “Team Project Name + \ + Build Definition Name”.  Finally it will add its four standard subfolders of: Binaries, BuildType, Sources, TestResults.

To the left, you can see how the build agents local file system would look like.  Now, the MSBuild evaluation issue… concatenate those paths together from the eyes of the build agent:

D:\Dev\Team Project 1\Application A\Main\Code\MyApplication\MyApplication.Business.Entities\MyApplication.Business.Entities.csproj/../../../../../../Team Project 2/CoreLib C/Main/Code/CoreLib.Domain.Entities/CoreLib.Domain.Entities.csproj (238 characters)

This already dangerously close to the 259 character limitation in Windows operating systems.  And most projects will have more folders underneath the project level to give more namespacing and structure.

Initial issues with all of this should be brainstormed as: “Why are we naming the CSPROJ the namespace? Why are the folders named the namespace?”  Those simple things that aren’t preached much would eliminate alot of waste in valuable character limits.

But of course, the most important: “Why am I adding a project reference to CoreLib C at all?”  It’s a shared/enterprise level library… It should be treated as an internal product that other software uses.

I will cover this in part 2.


Under the hood of a Gated Checkin

After squashing a few “hey this seems broken” statements after introducing Gated checkins, like most good programmers: I refuse to do anything twice… instead I will automate/document/write a batch file/etc.

Misconception: “Its not associating to any work items or changesets? Something isn’t right…”

Well, of course.  It’s not supposed to.  I naturally had to elaborate on this.

The process of which a Gated checkin takes place is quite simple and putting too much thought into Microsoft’s automagically pipeline will tend to give it the appearance of overcomplexion.

The changeset simply does not exist.  It won’t exist.  That is the sole purpose of the Gated checkin.  It creates a private shelveset automatically based on the developer’s “changeset” it intercepted.  TeamBuild will then queue the shelveset build.

At this point the build workflow begins as normal except one extra step at the beginning… after Getting Sources, it will merge the shelveset and then continue to let MSBuild compile.  At the end of the workflow, there is one final inserted step as well (I’m sure you see where this is going): If successful (compile and/or unit tests) it will merge the shelveset into the repository.

The reconciliation phase then begins…  if it failed, you pull the changes back down and correct it.  If it passed and was merged, you will reconcile your workspace (if you didnt preserve changes on the checkin) and your team members will be notified to reconcile as well (assuming you are using Build Notification via Power Tools).

At the end of this process, if everything passes and the shelveset is accepted, a changeset is finally generated and finally associated to the “now-public” gated build.  It essentially replaces the CI build.

The nightly build will still pickup that changeset and perform associations as well.

Gated Checkin Flowchart