Home / ubuntu / Questions / 112863
Accepted
wim
Asked: 2012-03-15 02:06:16 +0800 CST

rsync not working between NTFS/FAT and EXT

  • 772

I have music that I play in my car, from an FAT32 USB stick. The folder which I use to put songs on is stored on my EXT4 hard drive. I add/remove/retag songs regularly and occasionally want to rsync the changes to the USB stick. But for some unknown reason (maybe permissions?), rsync copies all the files every time rather than just changed ones. I am calling rsync like:

rsync -vrlptgD source dest

How can I make it work like I want it to (i. e. know when a file hasn't been changed and don't copy it)?

ext4 backup rsync ntfs fat32

5 Answers

  1. Best Answer

    Javier Rivera's answer works, but it takes quite long for rsync to check and compare all file checksums. I found that using the following option worked better for me:

    rsync -rtv --modify-window=1 /source /dest

    The --modify-window=1 switch allows for a variance of ±1s on the timestamps. With this option enabled, timestamp comparison will be more lenient and look over the minuscule time differences between NTFS/FAT and Unix file systems.

    Source (ger): http://www.kai-hildebrandt.de/tutorials/rsync.html

    P.S.: Please be aware that DST will cause full file transfers twice a year. See here for further details and possible solutions.

    • 23

  2. Timestamps in FAT32 are too different from unix ones to rely on them to check for file changes, you should use also the -c switch, it will force rsync to compare all the files to detect changes instead of relying in timestamps. It will work, but it's slower.

    Finally, there are a couple of options in your command that can't work with FAT32 file systems.

    • -l will preserve links, FAT32 has no concept of links
    • -p will try to preserve permission, again no permissions on FAT32
    • -t will try to preserve modification timestamps, there is only one timestamp on FAT32
    • -g will try to preserve group ownership, again not supported by FAT32
    • -D will try to preserver special files and devices, you now what comes here.

    As htorque comments, the invalid options will no hurt you, they just will do nothing. But you must add -c switch.

    This:

    rsync -vrc source dest

    should work (at least it works on my computer).

    • 13

  3. I was having a similar issue under OSX, and Glutanamate's answer didn't help. Some of the files differ by an hour; this might be because I tend to cross timezones relatively often. Other files are off by a day or even a month. I'm not sure why this is. Checksumming on some of the files with widely differing timestamps shows that they are, indeed, identical.

    In any case, it looks like the --size-only option, which tells rsync to ignore timestamps, will work for my purposes. -c / --checksum (as mentioned by Javier) also works, but takes a bit longer. I timed it and it took about a minute to compare checksums for the GB or so in the subdirectory I'm working with. Of course the speed at which this happens will be dependent on the slowest drive in the system; in my case, that's the SD card in my phone. However, that was after I'd already been doing some file manipulation (including checksumming), so many of the files may have already been copied into RAM cache.

    • 3

  4. You should also avoid using the popular -a option. My recommendation on FAT32 is

     rsync -vrc --delete --progress --no-p source  destination

    --no-p : no permission

    --delete : delete unmatched files and folder in destination(if you really want this)

    --progress : show progress during transfer. It is good for large files.

    • 2

  5. That's too many flags (-vrlptgD) you're using. Remember, rsync is a Linux utility and doesn't work with Fat32 and NTFS effectively.

    You'd have to hunt for tricks to be able to use it.

    Try:

    rsync -rvh --size-only --progress --delete /path/to/ext4/ /path/to/fat32/

    More info here

    • 1