My problem is to process images through ec2 instances.
Based this similar question: Separate server for Resque workers
And in this article: http://highscalability.com/blog/2009/11/6/product-resque-githubs-distrubuted-job-queue.html
I'm planning to have distributed workers around ec2 instances, lets say 3. But I havent figured out the right way to do this.
My doubts are:
- I have to install resque on each amazon ec2 instance?
- If yes, they have to share a single redis server or each one can have their own?
- Is that right to use SQS and each intance will listen this global queue and process my incoming files?
Thanks a lot if you guys can help!
You will need to install resque to each instance, but they can use a single Redis server to pull work from. The Resque pool-master on each instance will track the list of queues it's configured to poll, and will spin up workers based on what it pulls.