Home / server / Questions / 137442
Accepted
Jakub
Asked: 2010-05-01 07:22:01 +0800 CST

Distributed computing for a company? Is there such a 'free' thing?

  • 772

I am new to the whole distributed computing / cloud thing. But I had an idea at work for our multimedia stuff like movie encoding / cpu intensive things tasks (which sometimes take a few hours).

Is there a 'free' (linux?) way to go about using a Windows machine, and offsetting those cpu cycles for that task to say 10 servers that are generally idle (cpu wise)?

I'm just curious if there is a way to do this or am I just grasping at straws here. My thought is that a 'cloud' setup would achieve this, however like I stated initially, I am a total newbie when it comes to it.

This is just an idea, looking for some thoughts? Anyone achieve this?

cloud-computing process cloud distributed-computing

4 Answers

  1. Best Answer

    Corporately we use grid computing for things similar to this. A great one to investigate is Condor.

    • 3

  2. are those 'cpu intensive things'...

    • developed in-house? then you'll have to split and distribute the job yourself. there are some nice libraries, but everything is very low level.
    • prebuilt software? then talk to the developer/provider. either it's supported or it's not
    • same process applied to lots of individual items (for example, processing thousands of images)? then you might be able to set a simple system out of a DB table holding the subtask status + some batch scripts that: pick a task, mark as doing, do it, return the result, mark the task as done, repeat until all tasks are done.
    • 2

  3. I think that if you are talking about render farms, there are some examples of such projects.

    • 1

  4. Ubuntu with Sun Grid Engine would work and is available in the repos. I think everyone's ideas are going to require you to wipe the host servers though, not sure if that's what you're looking for.

    • 1