SID Duplication Bug in Virtual Server 2005 R2 "Differencing Virtual Hard Disk" implementation

So, here's a quirky one.

Microsoft's virtualization software - the Virtual Server 2005 R2 edition to be specific - includes a mechanism for reducing the size of files needed to create multiple Guest Operating Systems within Virtual Server. A "Guest", in MS terminology is the virtual environment (machines) that you create within the Virtual Server. The Virtual Server application itself is the "Host".

Guests are created as files (named with a .vhd extension) and are mounted within the Virtual environment as Virtual Hard Disks (hence the .vhd extension - capice?)

Now, imagine a scenario where you want to create three Windows Server 2003 machines inside Virtual Server. Ordinarily, you'd create each one and install Windows Server 2003 three times - one on each. In normal situations, Windows Server 2003 takes approximately 2 GB drive space. So, in your case, you'd need 6 GB of drive space to create those 3 servers.

Enter the "Differencing Virtual Hard Disk" concept. It's a simple concept, really, and it goes like this:

Since the underlying OS of our 3 machines is Windows Server 2003, and since we are talking virtualization here, we can save space by making all 3 machines use THE SAME base image. Why not install Windows Server 2003 once, save it and let each machine reference it as its base hard disk? No reason not to, except that we are likely to want to configure each server differently - like make one of them a DC, the other a File Server and the last an Exchange Server. In this case, you'd not be able to use just one image for obvious reasons. "Differencing Virtual Hard Disk" solves that problem. With differencing, we can create one separate hard disk for each machine AND link each hard disk to the main BASE IMAGE disk. Now, each different configuration we make to each machine is written to the corrsponding differencing disk, and because they are separate, all modifications to each of them do not affect the other. Why is this good? Well....... remember that ALL 3 machines are using ONLY ONE Windows Server 2003 installation - the BASE IMAGE. We do not need to install Windows Server 2003 three different times. Each differencing disk only contains the configuration we are making to each machine - a considerable reduction in physical hard disk space needed to host all three machines.

OK - I hope all that makes sense to you. I must admit that it is a great concept - except that ......let's just say it's buggy, and the bug does not readily manifest itself in an easy-to-discern manner, and that's my topic here. Apparently, differencing hard disks is still afflicted by the REAL-WORLD duplicate SID problem you frequently run into when you use disk-cloning/duplication tools like Ghost, ImageCast, RapidDeploy, etc. Microsoft recommends that you try not to duplicate disks WITHOUT the assistance of SysPrep. Furthermore, MS recommends that if you do run into SID duplication issues, go get NewSID or GhostWalker for assistance. See: http://support.microsoft.com/default.aspx?scid=kb;en-us;314828

Alright. With MS position on disk cloning and with differencing virtual disk being such an important element of Virtual Server, you'd think Virtual Server will NOT be afflicted with the SID duplication issue. Well... you'd be wrong.

Let's step through 2 demos of SID duplication issues in Virtual Server 2005 R2 "differencing virtual hard drive" implementation:

We are going to create one “Dynamically Expanding Virtual Hard Disk”. This is going to be the BASE disk to which our differencing disks will be attached. 

We call the drive W2K3-Base.vhd – very original, I know.

Here’s the first one – R2DC1-P1.vhd

 

R2DC1-P1.vhd is attached to W2K3-Base.vhd

Here’s the second one – R2DC1-P2.vhd

R2DC1-P2.vhd is also attached to W2K3-Base.vhd

We now create 2 Virtual Machines (R2DC1-P1 - using R2DC1-P1.vhd, and R2DC1-P2 – using R2DC1-P2.vhd)

We make R2DC1-P1 a Domain controller in a new Forest. We call the Forest “Play.loc"

R2DC1-P2 is still a stand-alone server.

We make sure that R2DC1-P2 is using R2DC1-P1 for DNS lookup, because we going to attempt to join R2DC1-P1 to the “Play” Forest. Down the road, we are also going to try and promote R2DC1-P1 into a DC in a Child domain in the “Play” Forest. So, let’s get our DNS house in order.

Act 1, Scene 1  - Now, we do the join.

We supply the necessary credentials.

Voila! Everything Works! Welcome to the “Play” domain, let’s play!

So, we reboot as required and now we want to log into the PLAY domain, just so we can access domain resources ……

Attempting to log into the new domain throws an error:

We log in locally and disjoin the computer from the domain:

Now we are back to workgroup, so we reboot and log in locally again:

Supply the necessary credentials for the existing Forest:

We are able to bind successfully and get a list of domains:

It even (very helpfully) gave us a valid NetBIOS name. Let’s accept it:

And we step through the wizard:

So far, so good. Nothing fanciful here. Just a bunch of wizards to step through:

No sign of impending troubles here:

Dutifully doing its stuff. It even claims to be successfully replicating the partitions:

Here comes the first and only sign of trouble:

It already exists? Hmmmm….. I don’t remember creating one . So, we go look in the usual places for any sign of tennis.play.loc. Of course it doesn’t exist anywhere because we never created one. Phantom error?

You’d think that clicking “Yes” here would fix something (anything). You’d be wrong.

It only makes dcpromo restart the process again.

And you end up at the same spot:

In the meantime, Event log is showing you …errrrrr…. nothing intelligent. In fact, it’s logging nothing relevant to the dcpromo process at this point. Nothing showing on the server we are promoting. Nothing showing on the existing server at the root of the Forest we are trying to join. Everyone is playing dumb here

.

So, three hours, forty tries, six gallons of coffee and four packs of antacids later, you give up and clicked the dreaded “No” button and, voila! You get:

AD was not installed. Now it's telling us. Duh! We know that now. We just wished it’d have said something earlier: