Home / server / Questions / 937587
In Process
Farhan
Asked: 2018-10-29 06:46:56 +0800 CST

R scripts that only run sequential, any way to parallelize its execution to multi cores?

  • 772

i have some R scripts, which only can run in sequential way, cannot be broken into chunks or any parallel library for R or any other language cannot be used.

Is there any way i can distribute the Sequential execution of code to multiple cores or may be multiple servers in network? to speed up execution ?

linux

1 Answers

  1. So let us assume you can run:

    ./my_script.R arg1

    And you want to run the script on arg1..arg1000. Then you can use GNU Parallel:

    parallel ./my_script.R {} ::: arg1 arg2 arg3 .. arg1000

    This will start one my_script.R per CPU thread.

    If you have multiple servers at your disposal that you can ssh to:

    parallel -Sserver1 -Sserver2 ./my_script.R {} ::: arg1 arg2 arg3 .. arg1000

    If this does not answer your question, please elaborate on your situation.

    • 0