Sam Elstob's questions

This is a question about Linux storage device names on RHEL5 and RHEL6

Most storage devices are accessible using device names /dev/sda, /dev/sdb, etc. By storage device I mean a device that could be partitioned, formatted and mounted as regular filesystem.

However, HP servers use /dev/cciss/c0d0, /dev/cciss/c0d1, etc. as the device name, probably due to historical reasons.

This means that commands or scripts need a special case to handle /dev/cciss/cxdx as opposed to /dev/sdx

I understand that the cciss module has been replaced by the hpsa module in RHEL6 which removes this inconsistency.

However, this is a more general question - are there any other non-standard storage device names that could trip up a command or script that is only looking for /dev/sdx?

I can be a bit of a stickler for correct terminology at work.

Recently we have been dealing with the effects of Large Receive Offload on our application. To quote wikipedia

It works by aggregating multiple incoming packets from a single stream into a larger buffer before they are passed higher up the networking stack, thus reducing the number of packets that have to be processed.

I have heard people at work refer to these as Jumbo packets. I am aware of Jumbo Frames in Ethernet. I think there could be confusion because Jumbo Frames are about what flows on the physical network - they affect the MTU, whereas LRO is only in the software layer and the MTU hasn't changed.

Is it correct to refer to any packet over 1500 bytes as a Jumbo packet, regardless of whether it was flowing on the physical network or not?

I will begin by stating that I do not believe this is a duplicate of Why is Enterprise Storage so expensive?.

My question is specifically about SAS drive enclosures, and justifying their expense.

Examples of the types of enclosures I'm referring to are:

• 1 HP D2700
• 2 Dell MD1220
• IBM EXP3524

Each of the above is a 2U direct attached external SAS drive enclosure, with space for around 24 X 2.5" drives.

I'm talking about the bare enclosure, not the drives. I am aware of the difference between enterprise class hard drives and consumer class.

As an example of "ball-park" prices, the HP D2700 (25 X 2.5" drives) is currently around $1750 without any drives (checked Dec 2012 on Amazon US). A low end HP DL360 server is around $2000, and that contains CPU, RAM, motherboard, SAS RAID controller, networking, and slots for 8 X 2.5" drives.

When presenting clients or management with a breakdown of costs for a proposed server with storage, it seems odd that the enclosure is a significant item, given that it is essentially passive (unless I am mistaken).

My questions are:

1. Have I misunderstood the components of a SAS drive enclosure? Isn't it just a passive enclosure with a power supply, SAS cabling, and space for lots of drives?

2. Why is the cost seemingly so expensive, especially when compared to a server. Given all the components that an enclosure does not have (motherboard, CPU, RAM, networking, video) I would expect an enclosure to be significantly less expensive.

Currently our strategy when making server recommendations to our clients is to avoid recommending an external drive enclosure because of the price of the enclosures. However, assuming one cannot physically fit enough drives into the base server, and the client does not have a SAN or NAS available, then an enclosure is a sensible option. It would be nice to be able to explain to the client why the enclosure costs as much as it does.

We are developing and deploying a data warehouse based on Oracle 11g EE with partitioning on RHEL5 64bit for a client.

The total storage required would be around 4TB of usable space.

I have been reading about balanced hardware configuration of data warehouses and that storage throughput is essential to the performance of a data warehouse; storage should be spec'd for sequential throughput not capacity, stripe and mirror everything.

Our customer is very keen on using their SAN (NetApp) for a number of reasons: e.g. centrally managed and backed up and they have already spent a lot of money on it.

It seems to me that even a relatively small number of local disks could have better throughput than a SAN e.g.

16 x 10k RPM 600GB SFF disks = 9600 GB raw = 4800 GB usable space using RAID-10

If each disk can produce 60MB/s throughput the aggregate throughput is 8 x 60 = 480 MB/s

My question: Is it possible to get >400MB/s sequential throughput to a single Oracle database host connected to a SAN?

I realize I'll need at least a 4Gbps or greater connection to the SAN. I see no theoretical reason why a SAN couldn't deliver this speed given enough disks.

Can a SAN deliver data at "data warehouse" rates in practice (at least 400MB/s)? Does anyone see these speeds in the real world? Perhaps there is some limit that I am not aware of that prevents seeing these kinds of rates to a single host via a SAN.

We have lots of Oracle and Red Hat experience but we are not that familiar with SAN. We are a small company and don't have one internally.