Home / server / Questions / 379489
In Process
SvrGuy
Asked: 2012-04-15 08:25:13 +0800 CST

Forcing rsync to convert file names to lower case

  • 772

We are using rsync to transfer some (millions) files from a Windows (NTFS/CYGWIN) server to a Linux (RHEL) server. We would like to force all file and directory names on the Linux box to be lower case.

Is there a way to make rsync automagically convert all file and directory names to lower case? For example, lets say the source file system had a file named:

/foo/BAR.gziP

Rsync would create (on the destination system)

/foo/bar.gzip

Obviously, with NTFS being a case insensitive file system there can not be any conflicts...

Failing the availability of an rsync option, is there an enhanced build or some other way to achieve this effect? Perhaps a mount option on CYGWIN? Perhaps a similar mount option on Linux?

Its RHEL, in case that matters.

filesystems rsync case-insensitive

3 Answers

  1. You can change the case of the resulting filenames on the target server after the rsync. I wouldn't attempt to do this mid-transfer (in case you need to restart the copy). As for making the change on the linux side, you'd need to determine if there are any conflicts. You will also need to determine if you need the directory names' case to be changed. Will all names be unique? If so, an appropriate find script coupled with the tr or rename command could do the job...

    # Examples - Don't run directly
`rename 'y/A-Z/a-z/' *` # would change case on files within a directory.
    • 2

  2. You can mount a case-insensitive file system. Look at this post.

    Also, this page suggests creating a disk image of type FAT32 and mounting it. The created fs will be case-insensitive such any Windows partition.

    Using such a solution will eliminate the need to convert all these millions of files to lower-case.

    • 2

  3. Not the most elegant solution, but you can use LD_PRELOAD to override the relevant system calls and force everything to lowercase. I thought it is fun so I did a little proof of concept and...

    > ls in out
in:
CyltApJik  keumyomDu  LidusIcweo  spydjiPa  SycsEyror  tusUngEg

out:
> rsync -av in/ --rsync-path='env LD_PRELOAD=$PWD/lowercase.so rsync' localhost:out/ 
sending incremental file list
./
CyltApJik
LidusIcweo
SycsEyror
keumyomDu
spydjiPa
tusUngEg

sent 372 bytes  received 129 bytes  1002.00 bytes/sec
total size is 0  speedup is 0.00

> ls out
cyltapjik  keumyomdu  lidusicweo  spydjipa  sycseyror  tusungeg

    And here is the sample, which may take a few iterations to become good enough to sync the whole thing.

    > cat lowercase.c

    #include <ctype.h>
#include <string.h>
#include <sys/stat.h>
#define __USE_GNU
#include <dlfcn.h>

static int (*real_lstat) (const char *, struct stat *) = NULL;
static int (*real_rename)(const char *, const char *)  = NULL;

char * lowered(const char * string)
{
        char * low = strndup(string, 2048);
        char * c;
        if (low == NULL) return NULL;
        for (c = low; *c; c++) {
                *c = tolower(*c);
        }
        return low;
}

int lstat(const char * path, struct stat * buf)
{
        int ret = 0;
        if (real_lstat == NULL) {
                real_lstat = dlsym(RTLD_NEXT, "lstat");
        }
        ret = real_lstat(path, buf);
        if (ret == 0) return ret;
        ret = real_lstat(lowered(path), buf);
        return ret;
}

int rename (__const char *__old, __const char *__new)
{
        if (real_rename == NULL) {
                real_rename = dlsym(RTLD_NEXT, "rename");
        }
        return real_rename(__old, lowered(__new));
}

    > gcc -ldl -fPIC -shared -o lowercase.so lowercase.c
    • 2