Home / server / Questions / 399240
Accepted
Andy Shinn
Asked: 2012-06-16 09:52:21 +0800 CST

How to prevent asymmetric routing with multiple eBGP routers?

  • 772

I have 2 routers announcing a /22 subnet to different providers (one providers connects to each of the 2 routers). I have split the /22 in two /23 to announce one /23 on each of the routers plus the /22 (the providers will take the more specific route). This allows me to fail over and keep traffic inside the /23 in and out the same provider.

What are other ways in which I could announce just the /22 with both routers and have packets from servers on the network behind the routers go back out the same router in which they came in from?

EDIT:

The main problem I come across, which end users and clients complain about the most, is that the least hop route is sometimes not the "optimal" route. In my case, I know that Provider B may have better latency to X nation. But when packets come in from provider B, they may go out Provider A or provider B. The reverse is also true. If I send a packet to X nation out provider A, even though it may have more hops back, the packet will likely come in from Provider B (which may have higher latency, packet loss, etc. to this nation)

routing bgp quagga ip-routing

3 Answers

  1. Best Answer

    Strictly speaking, you loose complete control of inbound routing paths when you announce your prefix to multiple providers because there are independent routing decisions made downstream for return traffic to you. Furthermore, your announcements could even be modified by downstream providers after you send them.

    Example

    This is one example of what can happen. Suppose you have AS 777, which owns 2.2.0.0/22. You have services that a company with Router A needs to access... Let's also assume that AS 100 doesn't have a good link to you (maybe it's intermittently corrupting traffic due to physical-layer problems you haven't been able to fix). So you think to yourself, "I'll just prepend all my announcements to AS100 with a large number of ASNs so nobody will prefer the AS100 link until I can fix this".

    bgp asymmetric

    The problem is that you only have complete control of your outbound routing decisions. You don't get complete control inbound... so let's suppose Router A's administrator doesn't know your link to AS100 is bad. They are dual-homed to AS200 and AS100, but AS100 offers much cheaper transit, per-Mbps; therefore Router A's engineer takes full routes from AS100 and only uses AS200 as a backup (taking only a default from them).

    As the admin of AS 777, you can force traffic to Router A through AS 200, but traffic from Router A to 2.2.0.0/22 would still take AS 100 (because the best route is through AS 100, at Router A).

    Possible solutions

    Usually asymmetric paths matter because of a load-balancer or firewall that is receiving the traffic. Some possible solutions:

    • For services offered in other ASNs, you could source NAT all outbound traffic to an address that is local to a link going to a specific AS (this doesn't work well for service providers though)
    • For services offered in your ASN, you could source NAT all inbound traffic to an address that is local to a subnet on your BGP peering router in question (this doesn't work well for service providers though)
    • You could set up a DMZ that receives traffic from all the different ASNs and offer a single ingress / egress point for traffic from your upstreams. Under some circumstances, you can even synchronize the state table on the DMZ devices. (this doesn't work well for service providers though)
    • If you are willing to use a features like BGP Conditional Advertisement, you can sometimes work around these kind of problems at the cost of not using other carriers for inbound traffic to 2.2.0.0/22.

    If you provide more details about the nature of the services and problem, we might be able to offer more specific advice.

    • 8

  2. What are other ways in which I could announce just the /22 with both routers and have packets from servers on the network behind the routers go back out the same router in which they came in from?

    Why do you want this? Or more accurately, why do you want to force this behavior (since typically this is what will happen thanks to the magic of routing)?

    The entire point of BGP and the distributed routing hierarchy that makes up the modern internet is that your packets will take the best available route to their destination. If a packet came in over Route A but Route B is a better choice for the reply for some reason then why would you not want to send your reply over the optimal route?

    The only reason I can see for this is wanting to prejudice your traffic away from a more expensive carrier (in which case there are better methods).

    • 1

  3. The way to achieve symmetry here is actually not so much a function of BGP as it would be NAT. Let's assume you continue with the scheme of advertising the /22 out of both routers as well as one /23 out of each. Set up outbound NAT such that traffic leaving a given border router is given a source address in that router's /23. To effect symmetry for inbound traffic you'd also need source NAT to translate connections from global sources to separate pools allocated to each border router.

    You will lose visibility at the server level into where the connections are actually coming from, but that information can be retrieved from the router via Netflow or similar mechanisms.

    There's a significant degree of potential complexity that's being introduced here that will also translate (no pun intended) to where- and how- you place default gateways for the servers. This might call for an intermediate tier of BGP speaking routers w/appropriate policies to keep locally originated traffic going in the right direction. In general, though, there's no elegant way to assure symmetric routing when you're advertising identical routes with identical metrics out of multiple routers - and, quite honestly, outside specific requirements for state-aware devices like certain firewalls or load balancers, this just isn't something that is typically a design goal for most networks.

    • 0