Home / server / Questions / 430467
Accepted
vlannoob
Asked: 2012-09-21 17:37:47 +0800 CST

Jumbo Frames, ISCSI and ESXi

  • 772

I have enabled Jumbo Frames (9000) in ESXi for all my vmNICs, vmKernels, vSwitches, iSCSI Bindings etc - basically anywhere in ESXi where it has an MTU settings I have put 9000 in it. The ports on the switches (Dell PowerConnects) are all set for Jumbo Frames. I have a Dell MD3200i with 2 controllers, each with 4 ports for iSCSI. Each of these ports is set to Jumbo Frames (9000) as well.

So now the questions:

  1. Do I need to log into each Windows Server VM I am running and delve into the NIC properties and manually set it to Jumbo Frames in the NIC properties in the device Manager as well?
  2. Whats the best way of testing that Jumbo Frames are indeed working as intended?
performance

3 Answers

  1. Best Answer

    Don't do this unless you know exactly what you're doing. Really only do it on your dedicated iSCSI NICs and connected switch ports and SAN NICs.

    There's really not many reasons to have non-storage ports set for Jumbo Frames with modern equipment.

    • 6

  2. In order to test if jumbo frames are working correctly:

    1. Enable SSH into the ESXi host and login to the shell(VmWare KB)
    2. Do a ping of a storage IP using the don't-fragment option and using a packet size higher than 1500, ex: vmkping -d -s 7000 storageipaddr

    If you receive something like:

    ~ # vmkping -d -s 7000 10.10.10.10
PING 10.10.10.10 (10.10.10.10): 7000 data bytes
sendto() failed (Message too long)
sendto() failed (Message too long)
sendto() failed (Message too long)

--- 10.10.10.10 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss

    It means that there is an issue with your configuration, Jumbo frames are not working! You should follow this doc to see if all of your virtual switches have proper MTU size.

    • 3

  3. Jumbo Frames are usually disabled by default on the NIC so you most likely will have to enable Jumbo Frames on the NIC and make sure that you configure Jumbo Frames to match the rest of your iSCSI network devices.

    As an aside, I've seen more than a few iSCSI implementations that experienced I/O problems that were originally attributed to Jumbo Frames (with the problem being thought to be that Jumbo Frames were disabled and needed to be enabled or that Jumbo Frames were enabled and needed to be disabled) that turned out to be Ethernet flow control problems. If you experience I/O problems on your iSCSI network the first thing I would do would be to look at the Ethernet statistics/counters on the iSCSI switches and look for a large number of Ethernet Pause frames. If you see those then your problem is related to Ethernet flow control and you should disable flow control on the iSCSI switches.

    • 2