Questions[sharding](server)

I've seen such design in my company.

A big mysql table is split into tens of smaller ones of identical schema on the same server in the same db, without any physical distribution, no innodb_file_per_table, no partitioning, etc. The data is distributed evenly and the read/write access pattern is uniform across all tables.

What's advantage of doing so performance-wise?

I am looking at adopting a Mongo cluster and am doing research. We hope to have a cluster that can scale from 10k updates per second to 100k updates per second with very write heavy traffic.

The primary question is, If we wanted to move to a newer version, is this possible without downtime?

Would we upgrade one shard / replica at a time and let it rejoin the group and hope it is backwards compatible and doesn't munge the data?

As a bonus, what is a cluster estimate on 10k updates per second? 100k?

I looked at http://www.mongodb.org/display/DOCS/Sharding+FAQ but couldn't find an upgrade procedure.

For a production sharded MongoDB installation we need 3 configuration servers. According to the documentation "the config server mongod process is fairly lightweight and can be ran on machines performing other work". However, in the default configuration, they all have journalling enabled, and with preallocation this takes up 3 GB of disk space. I assume that the actual data and transaction volume of a config server is quite small, so that this seems a bit too much.

Is there a way to (safely!) run these config servers with much less disk use for the journal? Do I need journalling at all on config servers? Can I set the journal size to be smaller?

I currently have a MongoDB replica set configured as such:

{
    "_id" : "ahspy_mongo_set",
    "version" : 13,
    "members" : [
            {
                    "_id" : 0,
                    "host" : "remotedatacenter.in.montreal.dns.com",
                    "priority" : 0
            },
            {
                    "_id" : 1,
                    "host" : "t1.micro.at.ec2.dns.com",
                    "arbiterOnly" : true
            },
            {
                    "_id" : 2,
                    "host" : "m1.xlarge.at.ec2.dns.com"
            }
    ]
}

So basically, one server at EC2 that backups up to another server in Montreal which, for multiple reasons, cannot become master.

I want to change my structure and I'm not sure where to start without causing a lot of trouble. In fact, I'm not even sure what I want to do is possible.

What I want to do is take the main DB server I have at EC2 (m1.xlarge) and turn it into 3 m1.large shards. So basically, go from:

• Replica at EC2 (m1.xlarge)
• Backup Replica in Montreal
• Arbiter

to

• Replica at EC2
  • Shard 1 (m1.large)
  • Shard 2 (m1.large)
  • Shard 3 (m1.large)
• Backup Replica in Montreal
• Arbiter

Is this possible? Can I shard the replica set in EC2 while keeping only one server in the replica in Montreal? If so, what's the proper way to do this? I'm having a real "Chicken and Egg" issue with Replication/Sharding in Mongo so any help would be appreciated.

Thank you very much :).

Which database to decide upon? Any comparisions?

• Existing: postgresql
• Issues
  • Not easily scalable horizontal. Needs sharding etc
  • Clustering does not solve the data growth problem
• Looking for: Any database that is easily horizontally scalable
  • Cassandra (Twitter uses that?)
  • MongoDB (rapidly gaining popularity)
  • Voldemort
  • Other?
• Why?
  • Data growing with snowball effect
  • existing postgresql locks table etc for vaccuum tasks periodically
  • Archiving data is tideous currently
  • Human interaction involved in existing archive, vaccuum, ... process periodically
  • Need a 'set it. forget it. just add another server when data grows more.' type of solution