I solved my Internet speed problem, but I'm still puzzled. I can't understand what happened.
My scenario: a small intranet is connected to Internet through a Linux gateway that connects to a Cable Modem (Motorola SBV5121), my provider gives me static IP and the nominal speed is 3Mbps/512Kbps.
The Linux gateway is an old Centos 5.10, with iptables and Dhcp server, with two NICs:
eth0 (Intranet) # 3Com Corporation 3c905C-TX/TX-M [Tornado]
eth1 (Internet) # Sundance Technology Inc / IC Plus Corp IC Plus IP100A Integrated 10/100 Ethernet MAC + PHY
[INTERNET]---[Motorola CableModem]----[eth1 LINUX-GW eth0]-----[LAN]---[desktop machines]
All pretty standard, and it was working for a long time; but lately I was experiencing low speed, especially when downloading. The usual tests (eg http://www.speakeasy.net/speedtest/ ) reported (consistenly) very low speeds (about 200Kpbs) when downloading from USA (I'm in South America); the speed was decent (though never reaching my 3Mbps limit) when downloading from my country: 1Mbps to 1.5 Mbps . The upload speed was right, near the 512Kbps limit. I had no problems with pings, no lost packets, both NIC reported a 100Mbps full-duplex connection. The tests were done from a desktop Windows machine, inside the LAN, but the behaviour was consistent with what I got when downloading from inside the Linux GW. Disabling iptables gave no difference.
I was pretty sure that it was a routing or shaping problem on my provider. On calling, they asked me to test the speed connecting a plain single Windows computer to the Cable Modem. I was extremely skeptical that it would make a difference.
[INTERNET]---[Motorola CableModem]----[Windows desktop machine]
To my astonishment it certainly did: the speed was right (3Mbps/512Kbps, whatever target).
I was very puzzled, I double checked everything, didn't find anything. The traceroute were the same. As a last resort, I tried switching the NICs in the GW. I didn't change the cables, nor the configuration, I did the simplest thing: I changed the MAC in the "ifcfg-ethX" configuration files, so that eth1 keeps connected to Internet, but the NICs are switched:
eth0 (Intranet) # Sundance Technology Inc / IC Plus Corp IC Plus IP100A Integrated 10/100 Ethernet MAC + PHY
eth1 (Internet) # 3Com Corporation 3c905C-TX/TX-M [Tornado]
[INTERNET]---[Motorola CableModem]----[eth1 LINUX-GW eth0]-----[LAN]---[desktop machines]
And now it works perfectly! How can this be?
Notice that it's not simply that the "Sundance" NIC has some problem, as in the current configuration, when I do the tests, from a LAN machine, I'm still going through the two cards as before (only that in different order). Furthermore, when I transfer inside the LAN (desktop <-> GW) it works at a good Ethernet speed.
Has someone a plausible explanation, or experienced some similar bahaviour?
The only explanation I can think of is the cable connecting the gateway to the modem is going bad.
Perhaps the baseband chip (the one actually doing the line encoding/decoding) is going bad, or is not that good from the start, so that the increase in line noise (due to degraded cable) confuses it significantly.
Try checking the error rate using ethtool; compare the results to the other NIC's. It might help shed some light.
One plausible explanation is a duplex negotiation failure between the sundance card and your DSL modem.
10/100Mbit ethernet was notorious for bad duplex negotiation. Many people recommended hard-coding the speed and duplex on both ends. I wasn't a big fan of this, but it was not unreasonable.
It has become less of a problem over the years. Gigabit ethernet auto-negotiation is reliable - in part, because there is no such thing as half-duplex gigabit. I imagine that the 10/100 autonegotiation has gotten smarter too with newer chipsets.