I'm working on an application that does network traffic generation. The application has eth1 in promiscous mode. So it directly handles all incoming and outgoing traffic.
One of the features I'm implementing is IP fragmentation and defragmentation. Incoming fragments need to be reassembled, and outgoing packets need to be fragmented if their sizes exceed the MTU of 1500 bytes.
One simple way of testing my application is by sending a ping command to the IP address of eth1:
ping -c1 -s 20000 10.3.2.1
This is working fine.
However, once the packet size exceeds ~53000 bytes it fails. According to Wireshark I receive fragments until fragment offset ~51000 and then nothing, followed by a reassembly timeout.
The maximum size of an IP packet is 65535 bytes. The ping command allows specifying a size up until 65507. This actually works if I ping to eth0 (OS controlled).
When inspecting the application from the inside with GDB all is going well. Stepping through the code reveals that the fragments enter my application, the IP packet is succesfully reassembled and then fragmented again, and the fragments are sent back to the sender. Even for the last fragment the return value of send(...) (socket API) is equal to the size of the fragment, indicating success.
Does anybody have an idea what could be going wrong?
Operating system is Linux (based on RTLinux).
If the packets are correctly fragmented then reassembled on client side, but the reverse does not work,
I would suggest first to try to inverse the roles (since the problem is on the way back) and check if, this time, the fragmented packets/reassembling can be performed by A
If not, there is a problem in the packets management from B to A, could be a firewall limitation, or any router or switch you may have in between.
If you concern the headers ... it could be the bound you are looking for :>
Also, do you have a constant window for TCP or it's adapting itself ? Maybe a smaller window is used and you can't send more data
Solved
The issue was packet loss. The IP response was sent to a switch over a gigabit line. This switch in turn forwarded the messages over a 100 MBit line. This means that packets arrived at greater speed than that they left, causing memory usage inside the switch to quickly rise. Once all memory was used the switch had no other option but to start dropping packets.