I run a site which has high traffic surges and because of that auto scaling solutions is very profitable for this case. Currently the web server is able to horizontally auto scale but the bottleneck is on the MySQL server.
- I have tried with Amazon RDS Multi-AZ but it takes like 15 minutes for the 12 GB database to upgrade with some minutes of downtime. It has helped a lot when I already knew that a traffic surge was going to happen in some specific moment.
- I have also considered Xeround. This is probably the best solution although it is quite expensive for databases of this size. Anyway it is not an option because I legally need the database to be in the European Union.
- I have read about Scalr but not sure if that could be helpful and how.
- I have seen that many cloud hosting providers offer vertical scaling solutions which I think it has 0 downtime (not sure if that is really possible, as far as I know they use Xen hypervisor). That could be a solution but I wonder if it has not downtime and how the MySQL config (and many other things on the OS) are able to upgrade also without downtime.
- I have tried with MySQL slave servers but it was not helpful at all.
- I am using memcache which helps a lot but it is not enough. I need to upgrade because of writes, not just because of reads.
Any suggestions? Thank you in advance
Actually, a simpler solution would be to try adding Memcached to your stack to save on DB load. This can drastically save load, and is much simpler than trying to solve the problem of standing up servers quickly (low difficulty), and then figuring a rapid MySQL sync (much higher difficulty.)http://toblender.com/?s=memcachedTo solve the problem of too many writes, the most common fix is adding memory to the server (s a larger working set can be kept in RAM), putting your DB on faster disks (SSDs are a good solution, but expensive), or sharding (which is expensive in additional servers and complexity).
Another way to reduce DB write load would be incorporating a an in-memory data store (such as Redis) to handle frequently-changing data, and if needed periodically write changes back to your main DB.
You should consider using a Star Topology
Here is what I am proposing
Prepare the Topology Like This
Step 01 : Setup 5 RSS with these common options
This will cause all tables to be created loaded as MyISAM Storage Engine
Step 02 : Setup DM and all RS Servers
tblname ROW_FORMAT=Fixed;
on all tables in RSStblname ENGINE=BLACKHOLE;
on all tables in DMStep 03 : Setup Replication From DM to all 5 RSS
Step 04 : Setup Replication From WM to DM
END OF SETUP
Here is how your Read/Write mechanism works
Now here is the catch...
service mysql stop
at the 5th RSSservice mysql stop
at the 5th RSSservice mysql stop
at the new RSSYou can use RSS #5 to spin up new servers over and over again
On a sidenote, please do not use XEROUND for the WM or DM because they do not support the InnoDB or BLACKHOLE storage engine.
I hope these ideas help.
If you use Innodb, you should consider Mysql multi-masters managed by Galera. It makes setting up mysql multi-master easier, and should make it easier for you to "semi" auto-scale.
If this is a application that you or you company is writing, you can consider moving to a sharded(partitioned) design for the application. But sharding can get complicated. Here is a link to get you started.
I am assuming you have "tuned" the mysql config files, as in allocated appropriate memory etc.
Is this in a rack that you control, or is it in the cloud? 12GB is a very small database relative to the size of disks that are available. Put it on a RAID1 or RAID10 array of small SLC SSDs and your write latency will disappear.
The Intel 311 series 20GB SLC SSD ($120 each) would do the job brilliantly.
If the database were larger, you could achieve similarly spectacular results by moving your database to an iSCSI target on a ZFS SAN server (built on commodity server hardware using Nexenta, OpenIndiana, FreeNAS, or whatever) and setting up a mirror of similar SSDs for your ZIL write cache. In all but the most extraordinary circumstances, Gigabit Ethernet is more than adequate to move database iSCSI traffic.