Home / server / Questions / 179108
Accepted
Chris Phillips
Asked: 2010-09-09 02:03:35 +0800 CST

virtual networking with iptables in KVM host

  • 772

I'm working on a design for a Linux based appliance and looking to use KVM for the virtualisation layer. The system has 3 VM's interconnected in very specific ways. The fag packet diagram I've been given looks like this...

       www
        |
+------ | ------------+
|       |             |
|      |A|            |
|       |             |
|       |             |
|      |B|-----|C|    |
|       |             |
+------ | ------------+
        |
       dbs

This basically mirrors the data flow - a web server at the front with a back end server behind it, and a subsidiary service on the side. A and C must be on "separate" networks due to internet proximity etc. This is fine for the bulk of the system, but this means that all management traffic for vm's A and C would need to pass through vm B to reach an external syslog server or such. Similarily for ssh access in, we'd need something like port forwarding on B. Whilst the volume of this traffic is minimal compared to the application traffic, it feels pretty horrible, especially as the automated building of these vm's using kickstart or some such, mean that we can't start building A or C until B is finished.

So I'm thinking about making it all a lot more confusing with something like this:

     www
      |
+---- | ----------------------+
|     |                       |
|     |       +-----------+   |
|     |       |           |   |
|    |A|-----|B|-----|C|  |   |
|     |       |       |   |   |
|     |       b       |   |   |
|     +-----a x c-----+   |   |
|             d           |   |
|             |           |   |
+------------ | --------- | --+
              |           |
             mgt         dbs

So the data flow between A B and C for the app is largely identical (but drawn flat here) and we add in a second management nic to each VM connected to another bridge which also has an interface on the host (a, b and c). iptables and forwarding is then used between these virtual host interfaces and the outside world (d) to ensure that whilst all machines can directly reach external services lower down in the environment, they can't reach each other in a way that means compromise of vm A does not reduce the security to any larger extent. I could probably use ebtables and a single bridge in my additions but I think it'd look too subtle with A and C being on the same subnet still.

It's a fair cop to note that there's a significant angle on what things look like on a diagram for management as well as the actual technical credibility of things (hence "separate" vs separate)! Simplicity is also a huge motivator, but at the cost of chucking all additional traffic through a 2nd VM feels real uncomfortable, as well as adding maybe 5 to 10 minutes for the build in a time sensitive environment

So essentially, am I nuts?

Thanks

Chris

networking kvm-virtualization bridge iptables

1 Answers

  1. Best Answer

    No, not nuts. But I do think you are making this sound more difficult than it needs to be. I've read your question several times, and there are still a few things I don't understand. What does machine C actually do? Why do you need it there? What do machines A and B do?.

    In any case, if I follow your first diagram, than all you need to do is apply simple IP routing and you're fine. If machine C forms a separate subnet with B (so that it is separated from the rest), then in order to get to it, routing must be set up like this:

    • on dbs: route all traffic through B (that's the default gateway, which knows how to route
    • traffic to C, because it is directly connected).
    • on www: route all traffic for C through A (only needed if A is not the default gateway)
    • on A: route all traffic for C through B (only needed if B is not the default gateway)
    • on C set B as the default gateway.

    Once the routing is in place, you can add firewalls to make sure that traffic only flows where you want it to flow. Unless you are experienced with this sort of stuff, I'd recommend to get the routing going first and then add the firewalls (one step at a time, so you always know what broke it. Because you will break it).

    • 0