Let's say I have 2 machines in the same data center but not necessarily in the same rack.
How common would dropped packets be when sent using UDP between these two machines?
I'm asking under the assumption that since there are only a few switches at most between the machines that the packets will not be dropped at all.
How common is out-of-order packet arrival within the same data center? My assumption is there's but one route 99.9% of the time so this cannot happen.
However, anytime I catch myself thinking in absolute terms I know I must be missing something!
What background information do I need to gain a better understanding of when to expect dropped packets, and how often they might be dropped, and arrive out of order for machines in the same data center?
Ultimately I'm trying to decide between using multicast UDP or PGM when communicating between different Linode VPS instances located in the same data center. The information must arrive and in order. Sure, UDP does not sound so great then!
But, if one can expect almost perfect or perfect delivery in the same data center, then it's fine. But, I am testing that assumption.
Thanks.
You can't rely on UDP to deliver packets in order because the specification doesn't provide those guarantees. Even assuming the most ideal situation, a single piece of ethernet cable between two hosts, there is still the matter of the OS, the network stack, the NIC driver, and the libc implementation that your writing against.
At every step in that chain, the writers of that code will have chosen NOT to prioritise ordering UDP packets even if they arrive in order for the simple reason that they don't have to.
One contrived example could be the data structure that incoming packets are read into, which might be a ring buffer. Packets arriving in order, will be placed, in order into the ring buffer, but it may be simpler for the driver writer to dump them to the upper layers of the networking code in memory order, hence randomising their ordering.
Taking your situation, a virtual machine run on a shared infrastructure that will be run for volume, not performance, then the probability of predicting the order UDP packets will be received will be low.
In short, if the spec says you can't rely on UDP packet ordering. You can't rely on it, and you can't try to tweak the environment to give a stronger guarantee than the spec ever promised.
If anyone is interested in experimenting, just use Wireshark. If someone really gets on our case about slow connectivity or dropped packets we just mirror a port on a switch, connect up a laptop with Wireshark and take a look.
Pretty much any switch will cause re-ordering of two packets at any time and is taken into account of by many network protocols such as PGM.
One thing to consider is that most data centres block datagrams and block multicast in order to simplify and reduce overhead of their network infrastructure.
The IP/PGM protocol itself only needs to be used if you have PGM Router Assist enabled and aware network elements between the server and clients, otherwise stick with PGM encapsulated within UDP and save the burden of managing application permissions.
If you use a messaging layer like ØMQ you can defer the choice of IP/PGM, UDP/PGM, or TCP overlay network to deployment and integration.
I wrote a simple .net multicast application that can measure dropped packets under specified pressure.
You can download it here: SimpleMulticastAnalyzer.
Enjoy.