Home / server / Questions / 229298
Accepted
Howiecamp
Asked: 2011-02-01 11:14:39 +0800 CST

How to automate an Amazon EC2 instance startup, execution of some commands and shutdown?

  • 772

I need to download 100 GB of files (it’s in about 150 files) within a 7 day period before they expire. The download is rate-limited by the host so it takes MUCH longer than the theoretical transfer rate based on normal Internet speeds. I have a script of curl http://curl.haxx.se/docs/manpage.html commands - one line per file.

I had the idea of automatically spinning up n EC2 instances, executing the command and FTPing the files to a central location, then shutting down the machines. How would I do this?

I don't care whether it's Linux or Windows.

amazon-ec2 amazon-web-services

4 Answers

  1. Best Answer

    I would recommend using one of Canonical's AMIs in conjuction with cloud-init

    https://help.ubuntu.com/community/CloudInit

    Look for the section about "User-Data Scripts" where you can use the --user-data-file to auto-launch a script on startup. Thus, you can just put all the commands you want to run in the script and put "shutdown -h now" at the end of the script.

    • 7

  2. I've used boto to script such things before. Easiest to spin up linux images then ssh into them to have them do work.

    • 1

  3. Not sure of the best solution. But if you are doing this on a regular basis and need something to automatically start and stop your machines, then you may want to look at this product: http://LabSlice.com (with the disclaimer that I am the CTO).

    • 0

  4. Vagrant with the AWS provider does that easily, and is multi-platform.

    Once the Vagrantfile is created, then you launch the instance with vagrant up, login to it with vagrant ssh, and destroy it with vagrant destroy. Super-easy.

    • 0