Home / server / Questions / 444111
Accepted
chx
Asked: 2012-11-01 08:53:50 +0800 CST

How to distribute torrent files?

  • 772

I am doing some distributed work with RackSpace cloud servers and I am using bittorrent to distribute my files. It works surprisingly well. However, distributing the torrent files themselves are not so nice. How would you go around doing that? Right now I just scp the torrent files to the servers, and of course I could write a script that copies it to sqrt(n) servers instructing each to again copy to sqrt(n) but that's a pita to work it.

cloud

4 Answers

  1. Best Answer

    Not knowing what exactly your problem is, I can recommend pscp from parallel-ssh as a tool to upload small files to multiple servers.

    You prepare a list of servers to upload to and let it know what to take locally and where to put it remotely. For example:

    $ pscp -h list-of-servers file.torrent /tmp/
[1] 02:11:22 [SUCCESS] 10.0.0.21
[2] 02:11:22 [SUCCESS] 10.0.0.20
[3] 02:11:22 [SUCCESS] 10.0.0.45
[4] 02:11:22 [SUCCESS] 10.0.0.19
[5] 02:11:22 [SUCCESS] 10.0.0.2
[6] 02:11:22 [SUCCESS] 10.0.0.5
[7] 02:11:25 [FAILURE] 10.0.0.3 Exited with error code 1
    • 23

  2. Given they're very small files typically couldn't you just have a 'dropbox' using HTTP with a cron'ed script doing a curl/wget with wildcard?

    • 15

  3. You could have the torrent clients check an rss feed for new torrent files or switch to magnet links instead of torrent files.

    • 8

  4. You might want to look into murder

    Murder is a method of using Bittorrent to distribute files to a large amount of servers within a production environment. This allows for scaleable and fast deploys in environments of hundreds to tens of thousands of servers where centralized distribution systems wouldn't otherwise function

    .

    Murder was/is developed by the folks at twitter, and they're using it daily to distribute files.

    • 2