Performance penalty of multiple IP's per network interface

If you want to measure the performance penalty, you would need to create an appropriate client/server setup. and run transfer tests to your liking.

But there would not be a performance penalty for additional IP addresses. There might be an increase in CPU usage if you have more than one MAC address configured or run the interface in promiscuous mode (likely, if you have a bridging setup) as the host would need to evaluate the entire traffic that is received on eth0, not being able to use the NIC's filtering on broadcasts and the "own" MAC address. However, the impact would be negligible in a correctly configured switched network - the switch will mostly forward only relevant traffic to the interface.

Use 2 physical ports if you already have the cable, NIC and switch ports available.

Switches have internal caches that have a fixed allocation per port. Use two ports = twice as much cache available. Some NICs have internal caches (per port if multi-port NIC).

Some broadcast traffic can be filtered by the switch, more ports = potentially less broadcast per port.

As traffic reaches maximum throughput, collision re-transmits will increase load, two ports = less traffic per port = less potential collisions.

In most configurations you should be able to run 16 or more IP's on a single NIC without any measurable difference. If you never fill the switch cache, use "noisy" broadcast or multicast protocols, or run the connections over 75% max rated throughput you won't see a measurably difference to justify the cost of NIC, Switch port, and extra cabling.

The best reasons to use multiple ports are:

1. security/network segmentation using VLANS, network addressing, routing and/or source/dest/protocol access control on a managed smart switch.
2. Debugging/protocol capture and analysis, reduces clutter, less filtering needed.

We usually have 8 to 12 lightly loaded virtual clients running per development virtualization hosting server with no network issues. (Single NIC, Bridged virtuals, 8-12 IP addresses per NIC).

Just setup the different scenarios, and then test using tools like netperf, iperf, or even simple FTP transfer of some huge files.

The basic answer is that you can have essentially unlimited IPs on the same interface. That's not a problem.

But a virtual switch may have some overhead. However, this is rarely an issue and thus you will find very little information online about that.

I haven't used VirtualBox, but am using the Linux software bridging stack in numerous production gigabit environments with OpenVZ without any problems or performance loss at all.

If you really want the performance, then just bond the two interfaces as LACP (802.3ad) pair...