So I'm just going to jump right in to the maths; My web host offers a server with 5x2TB drives, unmetered 1GB/s connection and can provision servers in 5 different data-centers, for ~$200/month.
If I got 3 of these servers and install OpenStack Swift on them, I have ~10TB (I know it'll be less than that but to keep the maths simple I'll just say 10TB) of storage with the same features as Rackspace Cloud but for $0.058/GB (Compared to $0.15/GB for Rackspace). Plus free, unlimited bandwidth. The servers could be provisioned in 3 different DCs for redundancy and for scaling new servers are active within an hour so we could scale up our storage reasonably quickly if we needed to. We'd also be using a CDN to deliver content, so yes there would be bandwidth charges but they would be external so irrelevant to the question.
Obviously it's only more effective if working with large amounts of storage (For, say 2GB it's a lot less efficient) but we have 7.5TB of backups on a RSC files account so our effective price per GB would be $0.078 (But that would decrease) compared to over $1000/month with our current Rackspace system.
So my question is, other than having to manage our own servers and putting a bit more effort into scaling, what's the difference between a self-managed storage system and something like Rackspace cloud, is it worth $/GB difference+bandwidth charges?
The big deal with cloud is flexibility, and you pay a slight premium for it.
If you need 5 TB all the time, buying your own hardware is more cost efficient (though you still need to factor in maintenance, warranties, replacement, etc). If you need 5 TB for a short amount of time, renting it through the cloud would be much cheaper than buying all that equipment.
The same is true of VMs and other computer resources. For instance, if you need a lot of processing power short time you might rent a bunch of EC2 instances. I know one big financial institution starting about 15,000 Amazon virtual machines every evening to do accounting calculations. The instances do not run for a very long time, making the process very cost efficient.
The point is really the pay-as-you-use scheme, as well as the new space provisioning speed, that is faster than you buying more disks and etc. You also have less to manage and can work SLA's and so on.
Also take a look at this answer and the video referenced on it to see more about 'whats the big deal with cloud'.
In addition to what's already been said, it's the last paragraph that sums it up. Managing your own servers and scaling has a cost which isn't built in to the raw numbers. If you're looking at it purely from a numbers point of view then in effect doing it yourself for 10TB you're saving $900 per month. If you can manage these servers for that price and you keep them heavily used then you'll save, otherwise you're better off (financially) with the cloud.
Granted there's still bandwidth on top of the cloud option so it depends how it's being used.