Home / server / Questions / 753793
In Process
Naryna
Asked: 2016-02-04 09:21:19 +0800 CST

Multiple Nics, multiple default gateways on a Server 2012R2 cluster

  • 772

This is really just a question about best practices as there seems to be a little confusion. The physical hosts of our Hyper-V cluster have 4 Nic interfaces relating to the 4 subnets.

Now, when running the best practices analyzer, it loves to complain that most of these interfaces can't talk to each other. This is because all but one of the Nics have no gateway defined in the static settings. I was under the impression that best practice was to define only the one gateway, but I could be wrong.

So, the question:

Is it best practice to define one gateway per server, per subnet or per nic (in a cluster environment)? And, What are the effects of defining multiple gateways on a cluster environment?

windows-server-2012-r2

3 Answers

  1. For every network that the cluster members have in common (having a NIC in the same subnet), the cluster will create a cluster network and enable cluster communication for that cluster network (cluster heartbeat traffic, CSV metadata traffic and redirected CSV traffic). The cluster validation wizard will attempt to communicate from all cluster networks to all other cluster networks across all cluster members (such that if any single cluster network is unavailable on any cluster member, the cluster can still communicate with that cluster member across the other cluster networks). The fact that it can't communicate from a routed cluster network to a non-routed cluster network is expected. What you should do is determine which cluster networks you want the cluster to communicate on and disable cluster communication on all the others. You can do this in the properties of each cluster network. For instance, your iSCSI network doesn't need to be enabled for cluster communication (and in fact, you don't want cluster communication occurring on your iSCSI network).

    If you assigned an ip address to each physical NIC (presumably in different subnets and presumably in different VLAN's) then that's why you have multiple cluster networks and that's why the cluster is trying to communicate across all of them.

    Are you using a dedicated NIC for iSCSI traffic? Do you have a dedicated NIC for LiveMigration traffic? How about for CSV traffic?

    Can you tell us how you have your Hyper-V virtual switches configured? Are you allowing the management operating system to share all of your virtual switches? How many virtual switches have you created? Have you created one for each physical NIC?

    • 4

  2. Having a single default gateway is best. It then becomes the interface where traffic is sent if it doesn't match the other three interfaces or if there are no explicit routes in the routing table.

    If you have traffic for a specific subnet that shouldn't go out the "default gateway" use the route command to update the routing table. Specifically route add.

    Specifying multiple default gateways will impact routing. Use the route print command to verify the system's routing table doesn't have any conflicting entries.

    • 2

  3. Let me see if I can explain this, while all other answers given are valid as well.

    so you have 4 NIC's, and therefore you have 4 subnets configured, as the mask of the subnets will determine the size of the network the server can send packets to (directly).

    If a situation occurs that the server wants to send a packet to an address which is not directly attached to the configured subnets, it needs a "gateway" to send it to.

    Every NIC properties has a field called "Default Gateway" this is a little bit misleading, and comes from a time where a server would usually only have a single NIC.

    One or more of the subnets which are configured may have a router (which is a gateway)

    So you need to determine what router to use if a packet need to be send to an address outside of the configured subnets.

    If you only have one router, on one subnet, it would be simple: on the NIC connecting to that subnet you put in the IP address of the router in the "Default Gateway" box. And you leave this field blank for all the other NIC's

    Once you have done that, all packets which are destined outside the 4 subnets will be sent to that router, as this router is now the "Default Gateway" for the host.

    More advanced: It will get a bit more complex if you have more than one router, on more than one subnet. If this is your case, then you need to add multiple gateways to the routing table using the ROUTE ADD command. And you do need to put a metric in to determine which one is the "Default Gateway" (which is the one with the lowest metric).

    Summary: So there is only one "Default Gateway" for a given host, it is the "Gateway" with the "Lowest Metric" in the "route table".

    Easy config: only fill in the gateway in one of the NIC's and leave the other NIC's blank.

    Advanced config: add routers as gateways specifying metric with the ROUTE ADD command.

    To Check: you can use the ROUTE PRINT command to check the route table to see how this is configured.

    HTH, Edwin.

    • 2