How can I stop one virtual iis site from drowning out all the others?

The first answer is to ensure that this website runs in its own Application pool. This will ensure this site runs in its own worker process, so any errors will not affect the other sites. It also means you can specify different settings for things like Identity, Performance and Pool Health management.

That said, doing this it not going to stop this site using up all the bandwidth available and preventing the other sites working. The easiest answer to this problem is to have this site listen on a IP that is on a separate NIC and network connection, to the other sites. But obviously this involves having 2 upstream connections to your server. If this is not possible then your are going to need to look at some way to balance the traffic, this is something you may want to do do at your router or switch.

You can (and should) set limits on bandwidth for any IIS server that has the potential for inhaling all the bandwidth of a server. In addition you should also (as previously mentioned) have the sites in their own app pool

To throttle the bandwidth used by an individual Web site

1. In IIS Manager, expand the local computer, expand the Web Sites folder, right-click the Web site, and click Properties.

2. Click the Performance tab, and in the Bandwidth throttling section, select the Limit the network bandwidth available to this Web site check box.

3. In the Maximum bandwidth (in kilobytes per second) box, type the maximum number of kilobytes per second that you want this site to use. This value must be at least 1024 Kbps.

4. Click Apply, and then click OK.