I've been setting up Amazon EC2 instances for an upcoming project. They are all micro instances, running Ubuntu Server 64bit. Here's what I've setup so far:
- Web Server -- Apache
- Database Server -- MySQL
- Development Server -- Apache & MySQL
- File Server -- SVN & Bacula (backups are done to S3 buckets)
Currently, there's only one Web Server, but eventually there will be more.
My first question is, what is the best, most secure way for Amazon EC2 instances to communicate between each other? Currently I'm using SSH, is that the best method?
According to Amazon, instances communicating between themselves using their Elastic IP addresses will be charged data transfer fees. However, instances communicating using their Private IP addresses can do so for free. Unfortunately, it appears Private IPs change if the instance is stopped and re-started.
So that's my second question, how do you make use of Amazon instances' Private IPs if they're not static?
I know that the instances probably won't be stopped and started very frequently, but still, if the IP address is in various config files, it would be a pain to have to go through them all and change it.
I'm primarily concerned about the Web servers, which will need access to the Database server and the File server, which will need access to all the instances when performing backups.
Note: I've never used Bacula before and I don't have it setup yet, but I'm assuming it will need the IP addresses of the clients to back them up.
Check out Eric Hammond's article explaining how to use Elastic IP addresses even from within EC2. This method does NOT result in any bandwidth charges because resolving the Elastic IP address (by name) from within EC2 returns the Private IP address.
http://alestic.com/2009/06/ec2-elastic-ip-internal
For more options, I have an article examining a few alternatives:
http://shlomoswidler.com/2010/06/track-changes-to-your-dynamic-cloud-services-automatically.html
Deploy your EC2 instance into an AWS Virtual Private Cloud (VPC). When you configure your VPC you will assign CIDR to all the EC2 instances in the VPC and the internal IP will be static.
SSH is a very good method for transferring data between different servers but if you're looking for something like a permanent connection (to a database for example) you can use any kind of encrypted tunneling software like
stunnel
Since there's no way to have static private IPs you can use some kind of automatic server deployment for this, there's several tools like mcollective, capistrano or func that'll allow you to register your new instance to a central designated server and generate actions on multiple machines based on that