Home / server / Questions / 177728
Accepted
lg.
Asked: 2010-09-04 05:08:33 +0800 CST

How do I copy one 500GB file from win2003 to lan/usb without impacting the source server?

  • 772

I have to copy (periodically) a MSSQL backup file large 500 GB to another pc (windows or linux) within lan.

The normal copy (or xcopy, robocoy, teracopy) command to a network share (or external usb disk) takes a lot of time (about 12 hours) and the server with the backup file become very slow (because there is only one RAID SAS controller).

An "idea" maybe to split the backup file into smaller parts (like torrent) and copy these parts concurrently, but I don't know how to do it.

My primary target is to copy without impact the source server.

Any idea or tool suggested will be appreciated.

windows copy

6 Answers

  1. Best Answer

    With windows, unfortunately, your options are a bit more limited. You could simply try setting up a FTP server (don't use the IIS one) and set a max bandwidth. This should effectively lower the impact on the server.

    In all honesty however, If you're running into a noticeable slow-down when doing a single-threaded copy of a single file... I suspect that there are other issues at play. If you're talking about copying from SAS -> USB (or vise-versa) and the lag is happening... There is probably some issue with the USB stack. The USB stack shouldn't affect system performance that drastically. If you are running into the lag when doing ANYTHING from the SAS volume... perhaps the firmware of your raid controller is b0rked... or you're using the wrong drivers... (or in some cases... a REALLY cheap SAS controller). It can also be the result of some anti-virus program (or multiple programs overlapping) causing the machine to work 4x as hard as necessary. (as a rule... only use 1 anti-virus program. More will just make your machine work harder for no reason.) Even having your volume fragmented all over the place can cause the symptoms you're talking about.

    I would first try to fix the problem rather than simply work around it.

    • 3

  2. By your description, the server cannot cope with the I/O load. I would be inclined to think that's due to some other fault as TheCompWiz said. Until/unless you fix/alleviate the cause of the problem (you think it's insufficient spindles or too slow a RAID controller) you will not get it copied across any quicker.

    As your foremost intent is to copy it without impacting the server, look at throttling bandwidth/copy speed, i.e. deliberately making it run slower. robocopy has an "inter-packet gap" option robocopy /ipg:N (where N is some number) which can be used to limit speed. http://technet.microsoft.com/en-us/library/cc733145%28WS.10%29.aspx. I understand teracopy does too.

    To clarify do you need to copy this file only once, or are you needing to copy a backup file periodically on a schedule? If it's periodically, for backup purposes, then as Tom said, copying only what has changed from one 500GB file to the next 'version' will save a great deal of time and put less load on the server. Perhaps if you explained your overall intent rather than the bit you're currently stuck you might get some useful ideas.

    • 2

  3. the only way to really make this go alot faster is to go to some kind of block level solution that would only ever copy the deltas.

    • 0

  4. Why put lots of individual files into one big one then try to split it up again?

    Backup the individual files.

    • 0

  5. Have you monitored memory usage on the windows server while doing the copy? I wonder if you're running into the "large files" issue.

    Have a look at Windows Server 2008 x64, Large File Transfers, and Memory Usage (and some of the other pages linked from there)

    • 0

  6. Other solutions: Use differential or transactional backup instead of full backup. - This will cut down on the amount of data you will need to transfer.

    CDP: I use R1soft http://www.r1soft.com/ This software has saved my ass so many times.

    • 0