I have been tasked with designing a system do handle SMS transactions. Transactions come in form of HTTP requests from a service provider. In the process of handling transaction, system in question will need to contact several other systems outside of my control, and do some processing. I can choose pretty much everything, ranging from the language of implementation to database that will be used. I would like this system to scale well and to be highly available, at least potentially. Based on the above, my questions are:
1) What language should I chose? I am full time C++ programmer, but C++ does not seem like a good choice at all for this system. I know Java, Python and PHP well, and have tried many other languages. I am gearing towards Java. Would you chose Java too, or rather go with something else (given the requirements)?
2) I think to scale well this system needs to process things asynchronously, and there should be an ability to add processing nodes. For that, Java Message System looks like a right choice, for example Apache ActiveMQ implementation of it. Is this a good idea?
3) Obviously this thing needs a database of sorts. There is growing NoSQL movement - should I be looking this way at all (for example Apache Cassandra looks very promising), or just use tried and true PostgreSQL or MySQL with InnoDB? Because it seems to me that this thing needs ACID.
Use whichever language and tools you are most comfortable with using. Mission critical projects are not the time and place to be learning a new language. I'm a python and C/C++ guy, so i'd probably choose one of these. Don't let my language choices cloud your judgement though ;).
That said, Erlang would be a good choice, scales well.. just add more hardware and it scales horizontally. Problem is, it's a bitch to learn. See earlier comment about learning on big critical projects.
The same kinda applies to database choices.. If you're familiar with MySQL, use that. If you know Cassandra inside out, then that might be a better fit for you.
The kinda thing you're doing cries out for a mechanism of preserving transactional state, and atomicity. So choose a language and database combination that allows you to fulfil these criteria.
Asynchronous operation is a must-have, especially when you want to scale. I'd definitely side with some kind of AMQP type messaging platform, RabbitMQ is my favourite choice, but that's based simply on what i've got the most experience with.
The main advantage of this kind of queue over a database, is that you get the ability to be event-driven on message arrival, as opposed to waiting for data to arrive, or polling the database on a regular interval to see if there's new data waiting. You can set message persistence as a message property, so that if the server reboots, the message has been persisted to disk.