I have 4 ESX 4.0 hosts (a,b,c,d), 3 SATA Lefthand iSCSI SANS,1 SAS Lefthand iSCSI SAN
Hosts a,b & c can see all 3 SAS sans and the 1 SATA san
Hosts a,b,c & d can ping all 4 storage devices
Hosts a,b & c can vmkping all 4 storage devices
Why can host D ping the SATA san, but it cannot vmkping it?
You don't have a vmkernel port on host d that is correctly configured. It might simply be that you have incorrect settings on the port(s) or vswitch(es) or it might be an issue on the physical switch port your iSCSI uplinks from host d are connected to. Check that the correct physical nics are bound to the right things too, it might simply be that some cables have been swapped around. And it might simply be that you have dodgy cables on those iSCSI uplinks. You could have a dodgy nic on the server - it's always possible.
For ESX 3.5 and earlier you need both Service Console (ping tests this) and vmkernel (vmkping tests this) visibility to you SAN ports for iSCSI to work. With ESX 4 the Service Console requirement no longer applies but without a working vmkernel port that your iSCSI Initator can use you wont be able to get connected to the iSCSI targets.
Assuming that all four hosts are very similar a good place to start is to check that the network configuration on host d matches the pattern on the others. The CDP info bubble on vSwitch uplinks can be very useful as it will tell you a lot about the physical switch and switch port each uplink is connected to if your physical switching environment supports CDP.
Are the SANS on the same subnet as the ESX hosts, or on different networks?
If they are on different networks, you should check if the correct default gateway is defined for the VMKernel port; and, in any case, check its IP address and subnet mask.
VMKernel's network configuration is distinct from that of the Service Console, so it could very well be that something is misconfigured and, while the SC is able to talk to those SANs, the VMKernel can't.
After reviewing the submitted suggestions above and not resolving my issue; I finally decided to move on to another issue for a while to let my brain refresh. While working on the other issue that envolved taking down the entire VMware environment, I noticed that host D which couldn't see the SATA SAN before was now able to see the SAN. However; host C could no longer see the SATA SAN. As a test, I rebooted host D and then refreshed host C's storage and it could see the SATA SAN again. Once host D was back online, I tried refreshing the storage on host D and it could no longer see the SATA SAN.
This lead me to researching it being a possible SATA SAN issue. There are several issues that have been fixed with some firmware updates; one of them being an iSCSI limitation. We have yet to perform this update, but HP has confirmed this should resolve the issues we are seeing. Thanks to all for your input.