A Hosts file, in case you need to ask, is a static file that maps a given IP address to a human-readable domain name. In a way, it is the grandfather of the modern DNS.

OK, as I was saying, in a Windows environment, the hosts file serve some useful purposes and entries in this file SUPERCEDE entries in any DNS server. In fact, the presence of an entry in this file precludes that entry from being investigated or searched further. Again, this has been (and continues to be) behavior in a Windows environment, and all available technical materials from Microsoft describes this behavior without any caveat.

With the advent of domain-hijack and other malpractices on the internet, the use of hosts files as mechanism for security (albeit a poor man's security) has taken on significant importance. There are tons of materials (including from Microsoft) that recommend the use of hosts files to safeguard your small network and protect your web surfing activities. Let's describe this briefly.

Suppose you have a small network and you allow your workers to get on their computers and access the internet once in a while. Suppose you hear that there is a 0-day worm or virus spreading wildly on the internet, originating from specific domains and you want to prevent your workers from going to those domains without investing in a proxy/firewall solution (either because you can't afford it or you don't have the technical aptitude to configure and operate one), you will put these domain names in your hosts file and give them some bogus IP addresses (usually the localhost address). You distribute this file to all the computers on your network and, bam! you just bought yourself a measure of protection. OK, there are other more complicated use of hosts files (ISA configuration in a split-brain DNS scenario is one), but I digress.....

The point is that we have always been told that the hosts file trumps DNS entries. The resolver in your Windows Operating System is supposed to look in the hosts file, and if it finds any entry in there, it is EXPECTED to RESPECT that entry as configured. IF the hosts file tells my resolver that www.superduperdomain.dom is supposed to be at 1.2.3.4, then that is what my Windows is supposed to believe. Period.

Well, imagine the consternation and surprise when it was recently revealed that Microsoft recently altered this behavior in favor of some of its domains. It has now become public knowledge that some Microsoft URLs are now exempted from the known and expected name resolution behavior in Windows. All windows XP SP2 and Windows Server 2003 systems will now no longer refer the following URLs to the hosts files for lookup resolution:

• www.msdn.com
• msdn.com
• www.msn.com
• msn.com
• go.microsoft.com
• msdn.microsoft.com
• office.microsoft.com
• microsoftupdate.microsoft.com
• wustats.microsoft.com
• support.microsoft.com
• www.microsoft.com
• microsoft.com
• update.microsoft.com
• download.microsoft.com
• microsoftupdate.com
• windowsupdate.com
• windowsupdate.microsoft.com

What this means is that if you find yourself needing to prevent your users from accessing ANY of these URLs by using the trusty hosts file method that has served us well for so long, you are out of luck. Micorosft has changed the code to silently ignore the presence of these URLs in a hosts file and to go directly and ask a DNS server, in direct conflict with all available technical documentation.

In the conversation to date, the only argument that has been advanced in support of this change is that Microsoft "probably" introduced this change in order to protect the end-user - by ensuring that they are able to reach Microsoft sites to download updates and patches. The argument justifies this thinking by pointing to the rash of hosts file hijack that have lately been perpetrated by malware writers. When the malware writers pollute the hosts files, then Microsoft needs a way to be able to still let the user visit microsoft sites for updates - so the argument goes.

I am a reasonable person, and reasonable arguments have the tendency to sway me more often than not. Unfortunately, the above are not only indefensible arguments, they are assinine - and that's being generous.

For one thing, the majority of the URLs on the list above do NOT have anything to do with patch and security updates. Besides that, a malware that can pollute the hosts file can ALWAYS overwrite the modified windows component that introduced this behavorial changes, thereby defeating the "purpose" of the changes.

For another, if we hold our noses and pretend that the patch update argument has some merits, this is anti-competitive. Given the huge sanctions slapped on Microsoft both in the US and in EU for anti-competitive and monopolistic behaviors, a reasonable person would have been able to deduce that, by using its position of power to hardcode its own domains into a critical component of the Windows platform, making it behave differently and against all available documentation, without extending the same privilege to its competitors, Microsoft is playing like a scofflaw again.

Microsoft is now well entrenched in the security service provider space (through major acquisitions and innovations) and it is in direct competition with the likes of Trend Micro, Symantec, NAI both for the home-user and the corporate markets. EVERY one of these other companies face the same hosts files hijack/pollution  problems, and none of them has the ability to hardcode their domain names into the core resolver component like Microsoft just did. This not only puts them at great disadvantage, it gives Microsoft undue advantage and undeserved competitive edge. Microsoft can now go to market and proclaim "our products can survive hosts files hijack, see how good we are?". That is soooooo unfair.

What is also mind-boggling is that Microsoft did this under the radar. As of this writing, I have yet to locate a document describing this behavior, and none of the people I have contacted (people who usually know these kind of things) are aware of any public documentation of this behavior.

One is now left wondering how happy Microsoft would be IF (just IF) one of these other companies provide an update that includes their own copy of the dnsapi.dll (where this change was snuck into) that carries its own set of hardcoded URLs, replacing the one provided by Microsoft. Would Microsoft cry foul? Would Microsoft have a leg to stand on in a court of law if it decides to challenge such an action? Would Microsoft be viewed favorably (having been found guilty on many occassions for underhanded anti-trust malpractices) in a court of law IF one of its competitors sues for remedy?

Would Microsoft be able to refuse a reasonable request to include other companies/people URLs in the component? And if not, will there be a certain point where dnsapi.dll becomes so bloated from so many URLs being stuffed in it that it becomes too useless for its original intended purposes?

I am not one to believe that Microsoft employees are stupid. I know too many bright people inside Microsoft to even remotely entertain that possibility. So, I am going to chalk this down to Microsoft doing something simply because they could do it. And that is more frightening than stupidity. It is more dangerous IF Microsoft starts saying "it is our code, we can do whatever we want with it". That argument did not work in the past. There is no reason to believe that it will work now. That behavior just gives more credence to the belief in some quarters that Microsoft is decidedly evil.