Home / server / Questions / 43171
Accepted
Dentrasi
Asked: 2009-07-21 11:53:22 +0800 CST

ESXi NAS configuration

  • 772

My client's setup is as follows.

Two Dell Poweredge servers, running ESXi, with 3 virtual machine spread across them. Each has about 2TB of space in SAS disk, which host the virtual machines. These are connected to a NAS via gigabit ethernet, and run nightly backups of the VMs, using ghettoVCB. In the event of one Dell server failing, the VMs will be booted onto the other one from the backups on the NAS, losing at most 20 hours of data.

The NAS I have is a Thecus N5500, with 4 1TB disks in RAID5. It can be configured either to use iSCSI, or NFS, with a filesystem of XFS, ZFS or Ext3.

Looking online, I've seen some bad reviews for iSCSI on ESXi, some people have complained of terrible performance, and as one of the VMs is a Terminal Services server, it needs to be fairly fast. Is iSCSI/vmfs better than NFS, and which filesystem would you use as a backend for it? Does anyone have experience with this NAS and any performance issues with any configuration on it?

Thanks,

~Dentrasi

Edit: ESXi is running from a USB stick on an internal port on the MB.

filesystems iscsi network-attached-storage vmware-esxi

6 Answers

  1. I have a Customer using an iSCSI SAN between two VMware cluster nodes similar to yours. I wasn't involved in spec'ing the solution, so I can't comment on what considerations went into deciding what to purchase (though I don't think that they actually did much homework). They're using a Dell NX1950 running Windows Server 2003 Unified Storage Management Edition with a DASD Dell MD3000 cabinet.

    My Customer is seeing adequate performance (25 - 28MB/sec sustained throughput to the iSCSI SAN) and we haven't gone to any great lengths, yet, to speed things up. Running iSCSI over a dedicated network is absolutely key.

    Here's a nice article on optimizing VMware 3.X and iSCSI that can give you some details about considerations re: iSCSI and throughput.

    One option that can give you better performance and multipathing capability over the built-in ESX iSCSI initiator is to run software initiators inside the VMs themselves.

    • 1
  2. Best Answer

    I'll bet most search results with the keywords ESXi and iSCSI assumes hosting VM's on SAN.

    In your case the VM's are hosted on DAS and the iSCSI store is for backup traffic only. The runtime requirements for running VM's over iSCSI and backing up over iSCSI are completely different. You won't find many articles describing ESXi performance over iSCSI file shares.

    It's complicated.

    I'd go for the simplest solution: NFS. Little if any performance negatives. Much easier configuration, troubleshooting and testing.

    But it's not only about the numbers is it? SCSI over IP is just so cool.

    • 1

  3. Are you using the embedded (e.g., booting from internal USB key) or full/disk version of ESXi?

    I have heard that iSCSI poor performance on ESXi is because the embedded version uses a Busybox-like environment, which supposedly implements an inferior iSCSI software/driver. The only way I've heard to improve performance is to install the full version of ESXi on the local disks.

    • 0

  4. I dont have any with THAT Nas... but I do have a VMware cluster that runs on an iSCSI NAS.

    Dell Ns500 (MS Storage server with iSCSI target) 6 1TB SATA drives in RAID5 4 Dell 2950s... 32 Gbs RAM 6x 1Gb NICS

    Have about 150+ VMs (win 2k3) on it... no speed problems. Have had as many as 200.

    The production Vs of of this cluster runs on EMC Cx4. I can really tell a performace difference.

    • 0

  5. VMWare on SAN in general achieves poor performance (relatively to the real thing) because of the way it rewrites SCSI commands. However you can still obtain much better performance than file sharing but it needs to be properly tuned. This is particularly important for Windows VMs to properly align your disk partitions to the underlying physical RAID.

    If the partitions boundaries doesn't match the actual RAID device block boundaries, each read and write in the VM becomes two read and two writes on the RAID : instant halved performance...

    Basically, you must create your VMDK properly :

    1. create the VMDK from VMWare
    2. Attach it to some existing windows VM
    3. N00b go away : this is for real men, and even on windows that means COMMAND LINE : fire your DISKPART.EXE

    4. Select the drive, and partition it with a large enough alignment factor like

      create partition primary align=64

    5. Disconnect the VMDK. Make copies of it and use them to install your VMs and for all your virtual drives.

    6. Profit!
    • 0

  6. Here is an article for conguration of iSCSI Target on windows and ESX Server. http://www.kernsafe.com/Article_Product.aspx?id=5&&aid=26

    • 0