With Windows, we can setup load balancing for arbitrary IP services just by installing NLB and selecting which nodes we want in the cluster. It does some MAC address changing, then has algorithms to decide which node handles an incoming request.
Is there a production grade equivalent for Linux? I don't want something that requires a front server to sit physically (network-wise) in front of the machines (like certain hardware LBs do, and Linux Virtual Server does, AFAIK). I want something that'll act similar to Microsoft's NLB -- pick N nodes, they all listen on the same IP, and divvy up the incoming requests.
see: http://lnlb.sourceforge.net/
Seems to be exactly what you are asking for.
That page summarizes it as: a common IP shared between all nodes (on a virtual interface). All you have to do is to bind on the virtual interface, the driver will do the rest.
It sounds very much like Windows NLB.
If you want a loadbalancing like NLB you should have a look at clusterip in iptables. -> http://security.maruhn.com/iptables-tutorial/x8906.html
It does exactly the same, only the failover part is missing, but maybe there exists some tools for this.
Here are a few that people find quite good in the Linux community (minus LVS at your request). I personally have only used HAProxy so your mileage may vary.
HAProxy
Balance
Only supports load balancing for http/https requests:
Pound
PLB - Pure Load Balancer
Round robin DNS would be the simplest solution, but maybe it's too simple for your requirements.
LVS is still the way to go -- you don't have to install it on separate hardware to get it to load balance incoming requests. Effectively, you still think of the load-balancer and backend services as separate things, but they just happen to be installed in the same physical hosts.