We have a directory of files (about 4GB) - and we'd like to synchronise between 3 locations across the internet. However, we're very conscious about bandwidth, so we need the following features:
- Schedule when synchronising should occur (e.g. 6pm - 8am)
- Specify a speed limit (e.g. 100KB/s)
- Don't synchronise when certain applications are running
We also need the following:
- Works on Windows and Linux
- Is free or not too expensive
Is there such an application in existence? We ruled out Dropbox because it's not free over 2GB, and Live Mesh because we can't set a speed limit or schedule.
Unison is close in that it will do bi-directional copying and runs on windows and Linux and is under the GPL. Unison is based on rsync so it can transfer pretty efficiently.
It doesn't implement any of the scheduling or bandwidth limiting features you want. Those other features could be implemented by combining unison with additional tools and/or scripts to automate it.
Rsync if you are using Linux/Unix.
For a Windows system you can use SyncToy from the PowerToys kit, or use Robocopy (from the Resource Kit for 2000-2003, incl. in 2008+ though) if you want to script it.
I believe there are a few versions of rsync for windows. Since you want a multi-platform sync tool, rsync may be your best bet.
One comment I would have is that this will require 3 separate paths to sync using a a tool like Rsync, and you could have some replication delays you aren't expecting. One way to mitigate this would be to have a "master" copy, and only replicate from/to that.
If you have the possiblity of using just Windows at all 3 sites for your sync needs, ultimately your best option might be DFS. You could setup VPNs using PPTP and configure a DFS root that replicates between 3 servers in the 3 sites. The new DFS-R protocol is supposed to be very efficient (similar to Rsync), and all your replication issues can be handled by the DFS engine.
JungleDisk -- you get 5GB for "free" (use the Rackspace Cloud, not S3, to avoid transfer charges) with their Desktop Sync subscription that runs all of $3 a month.
http://www.jungledisk.com/personal/desktop/features/
You can set up the client software on any platform and have "sync" folders that will live push/pull (using the cloud as the go-between) to your machines. $0.15 per GB after 5 so it's really cheap if you go over.
The product is owned/backed by Rackspace, a very reputable company (so it's not going to disappear overnight).
If you can, I would seriously consider using Dropbox for synchronization. It would be well worth the $99/yr to not have to worry about synchronization, failures, disk space problems, etc. It doesn't do the scheduling thing you asked about but is always-on, but you can throttle it. It does bit-level syncing also saving bandwidth. Depending on the funding of your project I would go with that if at all possible and just worry about your actual project and files.
rsync would work well, but you will need to set up all of the scripting and you will have occasional errors and have to monitor it to make sure everything is syncing right. I use this as well but it stops working randomly and I have to fix a little bit here and there to keep it running.
A program I use but it Windows-only is SyncBackSE. It does all that you mention, including bandwidth throttling and scheduling. I've used it at home for backups for a number of years and have been very happy with it.
Since you have 3 hosts these systems, other than Dropbox (which I believe just keeps the latest file), may have problems with it. What if a file changes on 2 more more hosts?
Rsync also has a windows version. It runs as a batch file you can schedule in your scheduled tasks. Works like a charm