Home / server / Questions / 505802
Accepted
rjt
Asked: 2013-05-08 13:22:43 +0800 CST

rsync not removing single quotes around array of strings --filter list

  • 772

Trying to safely retrieve files from heavily infected Windows XP and 2003 machines that came to us from an acquisition. I have a similar need for automating a script that would run on PXE booted DRBL machines to rsync to our server.

The following code results in single quotes wrapping the entire '--filter=''' parameter to rsync. Note echo strips the single quotes, but not rsync. What am i doing wrong? Would making every parameter an entry in an array help?


FAILS because wrapping single quotes:

'--filter='\''- "Default User/"'\''  --filter='\''- "Application Data/"'\'' '


When i run the same exact command but manually remove the wrapping single quotes, rsync runs fine.
WORKS:

--filter='\''- "Default User/"'\''  --filter='\''- "Application Data/"'\''

Have tried without and without --protect-args.


#!/bin/sh -x

PROTECTARGS=--protect-args
EXCLUDE=( '- "Default User/"' '- "Application Data/"' )
echo EXCLUDE="${EXCLUDE[*]}" 
FILTER=()
for X in "${EXCLUDE[@]}"; do
  FILTER=( "${FILTER[@]}" "--filter='${X}' " );
done;
echo FILTER="${FILTER[*]}"

if [ "1" == "1" ]; then
rsync --dry-run -vv --stats \
      --recursive \
      "$PROTECTARGS"  \
      "${FILTER[*]}"  \
      '/hddOld/Documents and Settings' \
      '/hddNew/'
fi;

OUTPUT:


FILTER=--filter='- "Default User/"'  --filter='- "Application Data/"' 
+ '[' 1 == 1 ']'
+ rsync --dry-run -vv --stats --recursive --protect-args '--filter='\''- "Default User/"'\''  --filter='\''- "Application Data/"'\'' ' '/hddOld/Documents and Settings' /hddNew/
Unknown filter rule: `'- "Default User/"'  --filter='- "Application Data/"' '
rsync error: syntax or usage error (code 1) at exclude.c(817) [client=3.0.6]





bash-4.1.2-14.el6.x86_64
rsync  version 3.0.6  protocol version 30
rsync-3.0.6-9.el6.x86_64
CentOS 6.4 x64

WORKS ... NO IT DOES NOT:


FILTER+=( --filter=""${X}""  );

RESULTS IN:
'--filter=- "Default User/" --filter=- "Application Data/"'

rsync does not complain about this filter rule but does not actually filter out "Application Data/".

bash

2 Answers

  1. You have two problems:

    First, don't use quotes in --filter (especially with --protect-args that says use this to avoid having to escape spaces, though my testing here shows that double quotes are taken literally without it too). The filter should be - Application Data/. As you have written it with the quotes it's looking for a file named " in the folder named "Application Data

    The other problem is that you want each filter to be a separate argument to rsync, so use @ instead of * in the command:

    rsync --dry-run -vv --stats \
      --recursive \
      "$PROTECTARGS"  \
      "${FILTER[@]}"  \
      '/hddOld/Documents and Settings' \
      '/hddNew/'

    Tested with:

    GNU bash, version 4.1.5(1)-release (x86_64-pc-linux-gnu)
    rsync version 3.0.7 protocol version 30

    I created an array with arr=(a b "c d"), then used touch "${arr[@]}" to create files

    a
b
c d

    in dir1.

    rsync -r --filter='- c d' dir1/ dir2

    copies a and b to dir2.

    rsync -r --filter='- "c d"' dir1/ dir3

    copies all the files since rsync is looking for a file named "c d" with quotes in the filename. Also tested using mkdir and - c d/ and - "c d/" to make sure directory filters did not work differently, and "c d/" did not exclude the directory while c d/ did exclude it.

    • 1
  2. Best Answer

    I got it to work by doing this:

    I changed:

    EXCLUDE=( '- "Default User/"' '- "Application Data/"' )

    to

    EXCLUDE=( '-_Default User/' '-_Application Data/' )

    This removed the double quotes and used the alternate underscore "_" syntax after the "-". A prior answer suggested this but I had to add the "_" syntax to get it to work for me.

    Also changed:

    FILTER=( "${FILTER[@]}" "--filter='${X}' " );

    to

    FILTER=( "${FILTER[@]}" "--filter=${X}" );

    This removed the apostrophes from around the X variable substitution.

    And changed:

    "${FILTER[*]}"

    to

    "${FILTER[@]}"

    This causes each element of the array to be treated as a single parameter, rather than the whole array as noted previously.

    • 1