3 WAN and a Cisco Router - PBR, QOS, Load-Balancing

There are two main approaches that you can take on this to get both objectives of inbound and outbound load balancing working in concert, which I'll summarize.

With BGP

Requirements: BGP capable router, AS number, IP ranges that you're authorized to announce

When using BGP to balance load between multiple links, you're likely relying on it for all of your inbound and outbound traffic shaping, balancing, and failover, or else potentially using other options for shaping outbound traffic such as Optimized Edge Routing (aka Performance Routing).

The configuration will essentially look like this:

So, in this situation, you have a public IP space (192.0.2.0/24) that each ISP is allowing you to advertise through their BGP peering session with you. Depending on the speed of the links, you can do a number of things to distribute the load as desired, but it's seldom an exact art.

For outbound traffic, you'll apply a route-map to incoming routes from each peer to adjust their priority.

For inbound traffic, you'll make changes to the routes that you're advertising to each peer; you can apply a path prepend (so that the path to your system through that ISP appears artificially longer and is less likely to be used), or a community string; a community string can allow remote influence of the behavior of another router for that route, but is specific to how it's handled in that AS - ask your ISP what strings to use for what you want to achieve.

Without BGP

Major caveat alert: with different ISPs assigning you different ranges, each assigned range is going to route to you over a different ISP; in essence, there's no effective way to expose a public service over an address that's accessible via all connections. From there, the workarounds are ugly at best; round-robin DNS across all of the public addresses and dynamic DNS services come to mind.

With multiple ISPs treating you as just a client with some statically assigned space, your options wane significantly. Outbound balancing is still semi-workable for certain definitions of "balancing," but the configurations get a lot uglier and a lot less flexible. Inbound traffic has no visibility whatsoever into your link's redundancy, really, no load balancing at all - your best option is to provide redundancy for internet services in another way; conversations initiated from a local device to an internet server will always be responded to on the same ISP link.

It looks like this:

Each ISP just takes the packets you send them and routes back packets bound for your IP range on their link. Your router is doing separate NAT of internal hosts to the appropriate address space depending on what link it's going out. A given TCP conversation must continue to use the same link or it will be lost, which limits your dynamic load balancing capabilities; best to just use policy routing.

From what you've said, you're strictly in the "no BGP" section. So, to specifically address your questions:

Policy based routing is the tool you'll want to use for outbound traffic, yes. You can use extended ACLs as well as standard in matching, which means you can define rules based on services as well as just IPs. You can also look into configuring SLA monitors for your routes, to drop the route for a down ISP.

Inbound traffic is trickier, if you have any internet exposed services, and depends on what you're serving - round robin DNS and dynamic DNS services are options, but they're not good ones.

You'll likely want to do your traffic management (QoS/rate limits) on the external interface for each ISP.

Cisco has a couple of good examples on implementation specifics for a setup like this, take a look here for a configuration very close to what you're looking for.

You're kind of all over the place here. BGP is a routing protocol typically used to advertise your AS (autonomous system) to the Internet so that your public addresses can fail over from one ISP to the next. Though it's mechanisms may be used in some load balancing techniques, this isn't really its purpose.

Policy based routing could be used to balance traffic by changing the next-hop gateway based upon another condition like the source of the received packet. eg. you could have one network use one ISP, another use a different one.

I suggest, since these are small DSL lines that you take a look at something like a Sonicwall firewall which has a very simple mechanism for load balancing. You also need to consider that some applications will be broken by load balancing since not every client to server request is guaranteed to original from the same WAN address.

Your Cisco router may have some more capabilities than I am familiar with for achieving load balancing, but the pitfalls are the same.

Try this link here: http://www.cisco.com/en/US/tech/tk648/tk361/technologies_configuration_example09186a0080950834.shtml