ISP provides 400MB connection but behind iptables firewall I only get about 150MB

I have COX communications as my ISP and I have a 400 MB connection at work. I have had for years an iptables firewall I setup that runs great.

CPU Info

Intel® Core™ i5-2400 Processor (6M Cache, 3.10 GHz)

I have 2 1000Mb/s NICs in it.

2 x DGE-530T Gigabit Ethernet Adapter (rev 11)

I use all CAT6 cables for connections

All my switches support Gigabyte.

Using ethtools I have confirmed their setup

iptables

About 50 lines of code (even after doing a iptables -F it still slow)

When I get in front of the firewall and connect directly to the cable modem I get the full 400Mb/s. When I get behind the firewall it slows down to 100Mb/s give or take, last night I saw up to 160Mb/s. I have tried all times of the day including late at night when no one is on and I always get the same results. I have installed other NIC's in it as well and still get the same results. I am thinking it is my box but I have a quad core which is by far enough power to handle this.

Is there something I am missing? Are there some other tools I can use to troubleshoot this?