I have a script which searches all files in multiple subfolders and archives to tar. My script is
for FILE in `find . -type f -name '*.*'`
do
if [[ ! -f archive.tar ]]; then
tar -cpf archive.tar $FILE
else
tar -upf archive.tar $FILE
fi
done
The find command gives me the following output
find . -type f -iname '*.*'
./F1/F1-2013-03-19 160413.csv
./F1/F1-2013-03-19 164411.csv
./F1-FAILED/F2/F1-2013-03-19 154412.csv
./F1-FAILED/F3/F1-2011-10-02 212910.csv
./F1-ARCHIVE/F1-2012-06-30 004408.csv
./F1-ARCHIVE/F1-2012-05-08 190408.csv
But the FILE variable only stores first part of the path ./F1/F1-2013-03-19 and then the next part 160413.csv.
I tried using read
with a while loop,
while read `find . -type f -iname '*.*'`; do ls $REPLY; done
but I get the following error
bash: read: `./F1/F1-2013-03-19': not a valid identifier
Can anyone suggest an alternative way?
Update
As suggested in the answers below I updated the scripts
#!/bin/bash
INPUT_DIR=/usr/local/F1
cd $INPUT_DIR
for FILE in "$(find . -type f -iname '*.*')"
do
archive=archive.tar
if [ -f $archive ]; then
tar uvf $archive "$FILE"
else
tar -cvf $archive "$FILE"
fi
done
The output that i get is
./test.sh
tar: ./F1/F1-2013-03-19 160413.csv\n./F1/F1-2013-03-19 164411.csv\n./F1/F1-2013-03-19 153413.csv\n./F1/F1-2013-03-19 154412.csv\n./F1/F1-2012-09-10 113409.csv\n./F1/F1-2013-03-19 152411.csv\n./.tar\n./F1-FAILED/F3/F1-2013-03-19 154412.csv\n./F1-FAILED/F3/F1-2013-03-19 170411.csv\n./F1-FAILED/F3/F1-2012-09-10 113409.csv\n./F1-FAILED/F2/F1-2011-10-03 113911.csv\n./F1-FAILED/F2/F1-2011-10-02 165908.csv\n./F1-FAILED/F2/F1-2011-10-02 212910.csv\n./F1-ARCHIVE/F1-2012-06-30 004408.csv\n./F1-ARCHIVE/F1-2011-08-17 133905.csv\n./F1-ARCHIVE/F1-2012-10-21 154410.csv\n./F1-ARCHIVE/F1-2012-05-08 190408.csv: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
Using
for
withfind
is the wrong approach here, see for example this writeup about the can of worms you are opening.The recommended approach is to use
find
,while
andread
as described here. Below is an example that should work for you:This way you delimit the filenames with null (
\0
) characters, this means that variation in space and other special characters will not cause problems.In order to update an archive with the files that
find
locates, you can pass its output directly totar
:Note that you do not have to differentiate between if the archive exists or not,
tar
will handle it sensibly. Also note the use of-printf
here to avoid including the./
bit in the archive.This works and is simpler:
Credit to Rupa (https://github.com/rupa/z) for this answer.
Try quoting the
for
loop like this:Without quotes, bash doesn't handle spaces and newlines (
\n
) well at all...Also try setting
In addition to proper quoting, you can tell
find
to use a NULL separator, and then read and process the results in awhile
loopThis should handle any filenames that are POSIX-compliant - see
man find
I think you may be better off using
find
's -exec option.Find then executes the command using a system call, so that spaces and newlines are preserved (rather a pipe, which would require quoting of special characters). Note that "tar -c" works whether or not the archive already exists, and that (at least with bash) neither {} nor + need to be quoted.
I did something like this to find files that may contain spaces.
Worked like a charm :)
As minerz029 suggested, you need to quote the expansion of the
find
command. You also need to quote all the substitutions of$FILE
in your loop.Note that the
$()
syntax should be preferred to the use of backticks; see this U & L question. I also removed the[[
keyword and replaced it by the[
command because it's POSIX.Most answers here break if there is a newline character in the filename. I use the bash more then 15 years, but only interactive.
In Python you can us os.walk(): http://docs.python.org/2/library/os.html#os.walk
And the tarfile module: http://docs.python.org/2/library/tarfile.html#tar-examples
I had a similar problem in a script I used to convert audio files. The file names had spaces, which caused issues for the converted file names. This solution worked for me on OSX, using zsh:
Since I was converting audio, I wanted to use the original filenames (including their spaces) for the new, converted audio files. The script I used includes FROM and TO parameters for specifying the audio formats. It also does some additional cutting in the loop to remove the extension. I was only interested in getting the complete file name, so I found it necessary to remove the extension before converting it using the TO variable.