In a test environment running in VMWare GSX 3.1 on a Windows 2003 host, we had problems creating a trust between two domains. The Windows 2003 domain controllers are SP1 running as VMs. I will tell you up front that the problem was caused by the fact that the domain controllers run as virtual machines with the VMWare Tools installed!

The following setup:

• Domain A: Windows 2000 domain controllers, native
• Domain B: Windows 2003 SP1 domain controllers, native
• DNS and WINS name resolution fully functional and verified
• You want a trust from domain B (2003 SP1) to domain A (could be any version)

When you try to create the trust with Active Directory Domains and Trusts on one of the Windows 2003 SP1 domain controllers (domain B), you will get an error about not being able to contact the domain controller in the trusted domain A. The error is shown directly after you specify the name of domain A.

You can create the trust with netdom.exe because you can specify credentials for both domains. However, verification of the trust using Active Directory and Trusts will fail. The trust will work though!

Under the hood, when you create a trust using the trust wizard, the Windows 2003 domain controller will first try to connect to the trusted domain's domain controller using the credentials from the trusting domain (domainB\administrator). Logically, that will fail (because there is no trust). However, after that failure, a null session should be setup so that the Windows 2003 domain controller can talk to the LSA using RPCs. The null session is setup by connecting to \\targetdcindomainA\ipc$ using null as user and password.

In our environment, the null session was not setup at all. After the wizard tries to connect with domainB\administrator and the (obvious ) error STATUS_LOGON_FAILURE as seen in a network trace, the process just stops. No null session is established, not even tried! And the trust wizard throws the error.

SOLUTION
Remove the Shared Folders feature of the VMWare Tools. That feature uses a driver, hgfs.sys, that apparently causes this behaviour. After I removed that driver, everything functioned normally. The null session was established and the trust wizard continued without issues. Verification of the trust afterwards is ok as well.

On some of my virtual machines, hgfs.sys was still loaded after removing the VMWare Tools. Manually check that the driver is not loaded.

This is not really an issue with virtualization. It is, quite simply, a driver issue. In production, I would recommend against using the shared folders feature.

NOTE
You should use VMWare GSX 3.2 with Windows Server 2003 SP1 virtual machines. I have not yet checked if GSX 3.2 solves these issues.