Active Directory Between Sites. Replication Problems

[Michael]

I have been experiencing problems with Acctive Directory between sites. I would appreciate if someone can help.

Site Layout: 1 Domain wgl.local with 2 sites.

Site 1: Ip Subnet 192.168.50.0
Dc1 (Windows Server 2003 Enterprise Edition 32 Bit. Global Catalogue Server and Bridgehead Server. Active Directory Domain Controller with AD Integrated DNS )
Dc2 (Windows Server 2003 Enterprise Edition 32 Bit. Active Directory Domain Controller with AD Integrated DNS)

Site 2: Ip Subnet 172.21.78.0
Dc3 (Windows Server 2003 R2 Enterprise Edition 64 Bit. Global Catalogue Server and Bridgehead Server. Active Directory Domain Controller with AD Integrated DNS)


We have 2 netscreen firewall connecting both sites via a static route based VPN. I have opened up all ports on each side of the firewall to ensure that no traffic is being blocked. I can ping to and from between both sites and regulary use RDP to connect to the Servers in site 1 with no problems whatsoever.

I understand that I had to ADPREP the forest of the 32bit domain controllers and that is what I did so that I could attempt the initial install of DC3. I did the ADPREP/forestprep via the use of the schema update from Microsoft (KB Article Number(s): 919151) on Dc1. Also, as I am already running the latest Service Packs on Dc1 there was no need to run adprep /domainprep well this is according to Microsoft's website (http://technet.microsoft.com/en-us/library/cc773360.aspx). I have also verified that the Schema was updated to R2.

With regards to my DC3 server in site 2. I ran Dcpromo and got the error message at the end of install ""Replication of the domain information was not completed due to an error, but will be completed after the computer is restarted"
I checked and noticed nothing was replicated between sites. I then did another attempt. I demoted the server via the force procedure as I was unable to demote it via the normal dcpromo procedure. I then via Ntdsutil did the metadata cleanup of that server from the main Dc1. I also ensured no old DNS records were left.

Once DC3 was fully demoted and a standard Server 2003. I renamed the server to a different name now called "server3". Restarted the server. I then changed its IP Address also and restarted the server once again and was no just a member of the "WORKGROUP" group. I checked the Event logs to ensure no errors were in place on the server. All OK.
I then did a windows server update to see if any newly released updates were available from MS. None required.
I then joined server3 to the domain to make it a member server. All Okay. I made it a secondary DNS server, all OK. The DNS zone transfer from DC1 took less than a few seconds to copy over. All well so far. From Server3, did an Nslookup of the Servers and even workstations in Site 1 via the Netbios names. All working fine. Even did reverse lookups to lookup an ip address so that it can resolve it to a name. All fine.
Likewise from Site 1, from Dc1, Dc2 and some workstations I did Forward and Reverse lookups for server3 situated in site 2. No problems.
I have checked the event logs on server3 and all is okay. No problems.
I then pinged the servers to and from site 1 and also from site 2. No problems.
Its now where I dread the problems to happen again.
I run Dcpromo from server3 in site 2.
At the end prompt after completing the process the same prompt comes up saying "Replication of the domain information was not completed due to an error, but will be completed after the computer is restarted".
On restart of server3, it is now an AD doman controller but no information was replicated from dc1!
If I go to the DNS MMC from Server3. It shows "X it cannot contact the DNS Server". If you look at the DNS Events it has an event id 4013. (The DNS Server was unable to open the Active Directory...)
If I go to Active Directory Users and computers MMC and connect to server 3 domain controller I can see no users or OU objects was replicated. I can not understand why not!!. I can connect to Dc1 and Dc2 Domain controllers from the MMC from Server 3 and see all the OU and containers.
I have run out of ideas and hope you can please help.
Is there any setting that you need to define under global policy to allow replication of AD between sites.
I can see that AD Replication is working fine between dc1 and dc2 in site 1 so was wondering is there anything that you need to set to permit replication or other between sites?
Your help would be much appreciated.
Michael

[Biggles77]

Have you had a read of this article? It may throw some light on your problem. Note too that there is a hotfix link in the top left corner of the article if it is required. http://support.microsoft.com/kb/919151/en-us

[Michael]

In respect of your link to the microsoft kb article. This article shows you how to prep your Active Directory 32 bit domain and forest infrastructure to accomodate the 64 bit schema. I have done this already and even verified that the new schema changes are in place. Had I not done this I would have not been able to get past the first initial installation stages of when the new 64bit server is promoted to a domain controller. Any other suggestions?

[DShack]

Michael, did you actually set up another site in the AD and assign your remote subnet to it so that the AD replication subsystem will know that DC3 is in that site and behave accordingly? Normally that shouldn't matter and is only for increasing efficiency of replication, but I wanted to check. Your procedure is flawless, afaics.

What does doing a DCDIAG on Server3 give you? And on DC1? Any other replication-related errors in the event log? Look through the event logs on all DCs for any warning events, not just error events, as sometimes initial replication can fail due to objects in the AD that are pending deletion and can't be replicated. Warning events will let you know about that sort of thing.

But DCDIAG is going to be your most important next step.

[Michael]

Hi Dshack, Thanks for your response.
With regards to the setup.
From AD Sites and Services. I have created 2 sites.
Site 1 (Original Default First Site Renamed) and Site 2.
Site 1 has two servers listed under it. Dc1 and Dc2.
Dc1 is the GC Server, The Licensing Server, The Bridgehead Server and listed under the NTDS Settings are the servers Dc2 and Dc3 which were both automatically generated. Transport Protocol for Dc2 is RPC and DC3 is IP.
Dc2 listed under the NTDS Settings has just servers Dc1 which was also automatically generated. Transport Protocol is RPC.

Site 2: has just 1 server associated with it. Dc3
Dc3 listed under the NTDS Settings has just servers Dc1 which was also automatically generated. Transport Protocol is IP.

Under Subnets I created 2 subnets 1 for each site and associated that to each site.

If I try to click the replicate now from Dc1 to Dc2 or Dc2 to Dc1 from Site 1. I get the prompt;
"Active Directory has replicated the connections"

If I try to do the Replicate now function from server dc3 to dc1 from site 1 or from dc1 to dc3 in site 2 I get the message;
"The Following error occured during the attempt to synchronize naming context wgl.local from domain controller dc1 to domain controller dc3: The naming context is in the process of being removed or is not replicated from the specified server.
The operation will not continue."

I can not understand why as I have gone through the settings again and again.

I have looked into the Event logs on Dc1 and noted the following;

Under Applications.
There are a lot of Event Id 1058.
Windows can not access the file gpt.ini for GPO CN...
There are also a lot of Event Id 1030
Windows can not query for the list of Group Policy Objects..

Under Directory Services
There are a lot of the eventIds
1311, 1865,1566.

Under Dns
There are a few EventID 4521s. I do not know if this is because on the DNS MMC on Dc1 I have all 3 Dns Servers listed. When I try to go to Dc3 Dns from the console you get the message "Cannot contact the DNS Server". I do not know if it this that is generating this error?

Under File Replication Service
There is 1 eventid of 13508 and 13509 between dc1 and dc2 but this was ober 10 days ago.

A majority of all of the errors in the events all started happening within the past week. I assume since I promted Dc3 within the domain.

Any further help would be much appreciated.

[Michael]

I have been looking into the Kb Article by Microsoft 839879. Even though I am not getting this EventId I thought I should still check the values.

From Dc1. I did the ADSI Edit as explained below;

Verify that the RID Master is replicating with another domain controller
If a newly promoted domain controller generates Event 16650, the domain controller may have obtained replication information from another domain controller that is not the RID Master. During promotion, the computer account for the new domain controller is modified. If these changes have not replicated to the domain controller that holds the RID master role, the request will fail when the newly promoted domain controller tries to obtain a RID pool.

To verify that the RID Master is replicating with at least one of its direct partners, follow these steps:1. Verify that the CN=RID Set object exists.

The CN=RID Set object is in the right pane of ADSI Edit when the domain controller is selected under OU=Domain Controllers in the left pane.

If no CN=RID Set object exists, you must demote that domain controller and then promote it again to create the object.
2. If the CN=RID Set object exists, make sure that the rIDSetReferences attribute on the domain controller's computer account object points to the distinguished name of the RID Set object, as shown in the following example:
CN=RID Set, CN=DC01,OU=Domain Controllers,CN=contoso,DC=local

If the rIDSetReferences attribute does not point to the distinguished name of the RID Set object, contact Microsoft Product Support Services for more information.

I noticed that Dc3 does not have any set objects in the right pane of the ADSI Edit .
Dc1 and dc2 both have. I have already demoted and promoted the dc3 server several times and always do the metadata cleanup.

Any further help or suggestions would be much appreciated.