Too many data replication technologies in environment

Which solution is better suited to a given scenario strongly depends on the applications and data involved, there is no "global" approach that can work everytime, everywhere.

There really are too many different scenarios involved. Some examples:

• Active Directory domain controllers are designed to work as a multi-master, replicated, distributed database; and you also need to have local DCs in a WAN setup, because domain-joined Windows machines need to have a DC available in order to work properly. At the same time, with AD you wouldn't benefit at all from having a lower-level replication, at the virtual server layer or at the physical storage layer, as you can't have a DC "mount" a database copied from another one, and you can't either start a cloned DC and expect it to work (don't even try to do that, USN rollback is nasty).
• For a file server, the exact opposite is true: while there are application-level replication technologies (DFS in Windows), for big amounts of data a storage-level replication is probably the best approach.
• Now, let's say you are using Exchange; recent Exchange systems (2007-2010) are designed with transactional database replication in mind, and this is the only officially supported approach; you actually can copy an Exchange database and mount it on another Exchange server (that's called "database portability"), so storage-level replication could be made to work... but it's a lot more painful, and moving users to such a server requires heavy reconfiguration. Also, while in the two aforementioned scenarios there actually is a point in having local copies of the data in multiple places (AD is needed everywhere, and users in different offices could need to access the same file server data), for Exchange this only makes sense in a disaster-recovery scenario, because in normal operation each user only accesses his/her own mailbox, which can only be active on a single server at a time.
• For various database systems, again, this could work both at the application level or at any underlying level; but this strongly depends on the application: if you need to modify data in multiple places (as opposed to only read them), there's no way storage replication can merge conflicting copies, and you will need to let your DBMS handle that.

Each technology has its own benefits and use cases; there really is no "one size fits all" approach.

I've so far considered two options. The over-arching idea of both is to assign responsibility for replication to one team only: either the system administrators or the application administrators. With only one team involved, you decrease the complexity of the system and the risk.

Standardize on Virtual Server Layer Replication

By configuring replication at the virtual server layer only, you can treat the operating system and any applications running in it as a black box. The system administrators are in control.

Advantages:

• The DR procedure for all applications is identical: stop the replication, start the DR virtual servers, and ask the different application teams to confirm that all is well.
• It is easy to confirm that you have complete coverage of all data.
• It is easy to plan server/storage maintenance because the involvement of the application teams is minimal.

Disadvantages:

• The DR servers are idle the majority of the time: a possible waste of server resources.
• It can be difficult to test DR without a full failover given that IP addresses match Prod.
• Replication is immediate, so if there is data corruption in Prod it is propagated immediately to DR.
• It may be difficult to use this method to clone to Dev/Test.
• Can't be used with systems that don't run well in a VM.

Can be used in combination with physical volume layer technologies to enhance performance of replication (while still being managed from the VM software).

Standardize on Application Layer Replication

By configuring replication at the application layer only, you can treat the server, storage and the operating system as a black box. The application administrators are in control.

Advantages:

• Many applications can run in a multi-master or master-slave mode to take advantage of all server resources.
• Many applications can transmit changes in a concise format, saving on bandwidth and improving performance.
• Some applications can delay the application of changes to prevent corruption from being propagated immediately at DR.
• It is relatively easy to sever replication and re-configure servers as Dev/Test.

Disadvantages:

• Each application team must configure replication in their own unique way.
• Not all applications support this kind of replication, so there needs to be a fall-back option.

Can be used in combination with filesystem layer technologies to support applications without the ability to replicate.