Home / server / Questions / 609886
Accepted
Fox
Asked: 2014-07-04 06:35:12 +0800 CST

Shell command to filter out non-existing files in a pipe

  • 772

I'm looking for a nice way to generate a backup tar.gz of what is going to be overridden by the extraction of another tar.gz.

tar -ztf patch.tar.gz | grep -v "/$" | tar -T- -zcvf backup.tar.gz

This works perfectly! However, sometimes, the new patch (patch.tar.gz) will not only contain files already existing in the HDD, but also may add new files which don't already exist. These files which don't exist will be impossible to backup, and the second tar will generate an error. The exit code is 2.

tar: folder/file.txt: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous error

I'm looking for shell command which only checks for file existence (stdin) et send existing files on stdout. Does it exist? I would need something like this:

tar -ztf patch.tar.gz | grep -v "/$" | filterfileexist | tar -T- -zcvf backup.tar.gz

I need one shell command and not a shell script.
Obviously, I could implement this shell command in C myself, because it's very simple, but I hope to find something generic which will be found on other UNIX platforms.

backup

3 Answers

  1. Best Answer

    Try use perl oneliner for this

    like

    cat filelist | perl -ne 'chomp(); if (-e $_) {print "$_\n"}' | tar -T- -zcvf backup.tar.gz
    • 6

  2. Here are a couple of alternatives.

    tar -ztf patch.tar.gz | grep -v "/$" | xargs -i sh -c 'test -f {} && echo {}' | tar -T- -zcvf backup.tar.gz

    or

    tar -ztf patch.tar.gz | grep -v "/$" | tar --ignore-failed-read -T- -zcvf backup.tar.gz

    The second example will exit with status zero after an attempt to backup a file that does not exist (tested on Centos 6.5), which avoids the problem.

    • 4

  3. Use find with xargs. Default behavior of find is to print filename to stdout if the file exists, non-existing files will have an error message printed to stderr

    $ echo exisiting-file missing-file | xargs find 2> /dev/null
./exisisting file

    If there are directories, add -type f or -maxdepth=0 to prevent find from default recursion.

    $ echo exisiting-file missing-file dir | xargs -I% find % -type f 2> /dev/null
./exisisting file

    The usual whitespace caveat applies. Safer version

    $ echo existing-file missing-file 'annoying
name sudo rm -rf /' | tr '\n' '\0' | xargs -0 find 2> /dev/null
    • 0