Home / server / Questions / 888876
In Process
Jared Dunham
Asked: 2017-12-20 07:22:52 +0800 CST

DigitalOcean's object storage, Spaces, is randomly slow on retrieving small files

  • 772

We switched from a custom block storage solution to DigitalOcean's object storage and I'm kind of disappointed in the speed.

Is there anyway we could retrieve objects asynchronously? Also some objects that are around 50 KB can take from 57 ms to 3 seconds... Is this normal for object storage solutions or is this just an issue with DigitalOcean?

enter image description here

Update from DigitalOcean:

After battling with support for the past week, I finally got an acceptable response:

"Our Spaces team are aware of some possible performance bottlenecks and are working on addressing those." - Mike G., Platform Support Advocate

I did notice, a day after I submitted a support request (2017-12-18 ), the average speed was 0.74 seconds (with an avg. file size of 78.6 KB) before the NYC3 storage went down. Now it's at 0.42 seconds with an avg. file size of 88.48 KB, tested on 2017-12-20.

digital-ocean

2 Answers

  1. 1) Object storage isn't designed to handle the workload you're intended to use it for. Making a long story short: Hash-based K/V seeks won't ever reach performance of a block device "immediate" seeks thru LBA addressing.

    https://cloudstore.interoute.com/knowledge-centre/library/object-storage-use-cases

    2) Digital Ocean is just @#$^E%^# storage system! Drop it like it's hot, and replace it with COTS hardware combined with Ceph.

    http://ceph.com/ceph-storage/object-storage/

    • 5

  2. randomly slow, good luck troubleshooting object storage performance - Digital ocean could be throttling you, the object storage public endpoint is ...public. It could also depend on your out going proxy server or ISP, anyway I would not expect it to be anything like as performant as custom block storage solution. Object storage is a 'fast enough' key/value system

    • 0