I am designing the next evolution of our IT infrastructure. I am considering implementing an iSCSI SAN. My current plan is to connect only our servers to the SAN, and then implement any file sharing to the desktops via network shares using one of the servers as a file server. After more reading, I am wondering if I need to implement a file server at all if I have an iSCSI SAN. Here are my questions:
Would I compromise the performance of the SAN to the servers if I let the SAN traffic run over the whole network, instead of just between the iSCSI target and the servers as clients?
If I create a "disk" on the iSCSI target, can multiple clients access the same "disk"? If so, can this replace network file shares?
I realize that question 1 is a loaded question, since I did not specify the particulars of my network, but I am just looking for general thoughts about heading down that path. Advance thanks for your thoughtful answers.
I remember wondering these exact questions, too.
Here's the scoop: A server sees an iscsi slice (called a LUN) just like it would a hard drive. It addresses it as /dev/sdwhatever, and you partition it and use LVM (if you want), and create a filesystem on that device. It works great.
Now, it gets complex when you want multiple machines to talk to the same drive at the same time. It would be like hooking a USB drive up to a bunch of computers at once. Madness. Cats and dogs laying down together. Insanity.
Now, it happens that there are some ways to get multiple machines to talk to the same device, but you have to use what's known as a "clustered" filesystem. This is a filesystem that knows that multiple computers are talking to it, and it accounts for that. It does this through a few possible ways, including multiple journals (one per machine, usually), or using a lock manager (which is one specific computer that acts like a traffic cop), but whichever way you go, you're going to have to cluster all of the machines that you want to talk to the same LUN.
What most people typically do, if they want really high availability, is to cluster three (or sometimes two, but it can be harder) machines to work together as a file server cluster. Those three machines are the only ones who write to the LUN, but their clients use NFS, Samba, FTP, or whatever to access the files there.
I actually did a Redhat Cluster HOWTO a while back (http://www.standalone-sysadmin.com/blog/2009/04/howto-redhat-cluster-suite/) but I never got it successfully working and stable. Now I've got a couple of machines that are set up as fileservers that don't have the LUN mounted, but can at the touch of a button. It's a trade off for me, since I didn't have the time to learn the clustering suite.
If you share the NIC for both iSCSI and normal traffic, yes, however look at the actual traffic you're sending to get a real answer. If there's no router between the servers and the workstations the broadcast overhead of the workstations would hinder performance slightly.
VLAN's are free, use one to seperate iSCSI traffic, most servers have 2xGbe these days, dedicating one for storage usually isn't a problem.
No. That's a NAS (and perhaps what you should be looking at, or at least a combo device like a NetApp).
There are "cluster" filesystems that do this, however they require all the hosts to be trusted and there's a lot of manual configuration involved.
You will also want to look at "zoning" (as it's called in the FC world) to ensure Windows (which will take anything it can get) doesn't steal another host's LUN (think partition)
1) If I understand you correctly: depends on what's best for you. Do you need multiple servers to access the same files? Both work fine with what they do, so this depends on what you need.
2) With specific file systems (GFS, OCFS) yes. else no.
We use NetApp hardware for our SAN and it gives us all the options
plus
This has greatly simplified our storage. I am in the process of migrating 2TB of disk from a VMware virtual machine Windows 2003 file server to being directly shared from the NetApp itself. Add in the ability to do snapshots and NDMP backups directly to tape, it is something to consider.
Sharing the SAN directly to desktops is probably more trouble than it's worth as you will have to install special cluster file system software on the desktops. This software can be quite expensive for Windows and is almost certainly not going to bring you any beneift. You are better off using a file server (it can mount SAN volumes if desired) and standard CIFS shares.
Some SAN equipment can also work in a dual mode where it exports both block level storage and NAS using conventional file sharing protocols. NetApp is best known for this type of equipment although you can get systems of this type from most manufacturers of such equipment.
As others have mentioned, iSCSI makes shared-access a lot harder. You need special file-systems for that (OCFS, etc) and performance on those file-systems is below that of others due to a variety of reasons. Such as not being able to use memory for caching as that prevents a consistent view of the file system on the multiple hosts, having to implement a file-locking system on the file-system itself rather than in-kernel, so performance can be significantly slower. It also just can't be done in Windows, as there are currently no multi-mount file-systems for that platform.
However, if you just want to present a slice of iSCSI disk for local access to the workstations, and handle shared-access storage through a file-server with iSCSI mounted storage then that would work well. The file-servers of NFS, CIFS, or Windows are designed to handle this sort of operation in memory rather than on the file-system so performance can be much higher.
Well there is Softwere that may be useful to your scenario, the StarWind iSCSI SAN: is software that converts your Windows Server into a great scalable SAN. This are some of his more interesting functions:
Has no restriction on individual disk capacity, total number of hard disk installed, number of CPUs or CPU cores, Ethernet ports or the amount of RAM.Convert any 64-bit or 32-bit Windows server into a SAN.
Unlimited connections and unlimited TB storage capacity. Supports Windows server clusters for high availability.
Virtualization environment support for VMware, Hyper-V, XenServer, Virtual Iron. Supports advanced VMware features: VMotion, Storage VMotion, HA, DRS and VCB. Shared storage for any server application including SQL Server, Exchange, SharePoint.
I will recommend you to use first the free try and then you see how it goes.
www.starwindsoftware.com
I would suggest you have a look at Solaris10 + ZFS as this can basically do all a Netapp filer would at a fraction of the cost (iSCSI, RW snapshots, CIFS, NFS, clustering and replication), the OS is free to use and it does not require any special hardware (most Dell/HP x86_64 would work just fine).
Cheers,
Simone