Home / server / Questions / 141809
Accepted
thor
Asked: 2010-05-14 23:51:06 +0800 CST

How to organise storage for media content such as video and music?

  • 772

Currently, we have a single server hosting all content: music, video and software. This content is downloaded by users through HTTP. Now free space is coming to an end and we are exploring different ways of extending our storage capacity. We want to do it cheap, simple and reliable (protected from disk/ server faults). Currenly, we see two ways:

  1. Add a couple of cheap servers with 4 disks (RAID1 ?), run some distributed file-system on top, like GlusterFS. Pros: hopefully, we will see all our disks as single flat file system, just dump content into it and be done. Cons: could be tricky in configuration and handling of faults.

  2. Add a couple of cheap servers, all running HTTP servers. Each piece of content (be it a music file or video) is placed on randomly selected two servers. Pros: don't have to deal with RAID, as content is duplicated; single server failure does not bring down any part of content; doubled distribution capacity (as any signle file could be downloaded from any of two servers hosting it). Cons: requires some scripting on part of distribution of content, adding/ removing servers.

Do we miss any other ways? Which of the aforementioned options seems to be the best?

linux glusterfs network-storage

2 Answers

  1. Never thought about opensolaris and zfs? You can export it via nfs/iscsi/smb, its pretty cheap to build a machine for opensolaris and the storage is easily expandable. Plus a easy way to back it up. Give it a try

    • 1
  2. Best Answer

    BOTH options are not really cheap if you go into the overhead. I would go for (1), but not with a 4 disc system - I would go for for a SuperMicro case with 24 discs (!) in 2 rack units. powered by a nice RAID controller that can handle more cases. I would then make RAID 5 groups out of 8 discs each, keeping the first 8 set for OS (2 discs, mirror), SSD caching (ok, using ADaptec Raid Controllers) (2 discs) and hot spares (4 discs).

    Depending on your needs you will find that to scale a LOT cheaper than low end servers where you need another server for every 4 discs, which I find ridiculously inefficient financially.

    That is, unless you expect your 4 discs to last forever.

    • 0