Home / server / Questions / 79813
Accepted
Phillip B Oldham
Asked: 2009-10-31 01:45:48 +0800 CST

`outputscript | gzip > $file` not overwriting $file

  • 772

I have an hourly cron script which take some output (a mysql dump), pipes it through gzip, and aims to overwrite a file of the same name. When I run it manually as root the file is overwritten. When it is ran by the cron daemon the filename has ".1" appended to it. This keeps happening so that after a while I have lots of files like so:

myfile.gz
myfile.gz.1
myfile.gz.1.1
myfile.gz.1.1.1
myfile.gz.1.1.1.1

and so on.

ps aux|grep crond shows that the daemon is being run as root.

I've tried:

  • renaming the original file, pushing the output, then removing the old file on completion, and
  • deleting the original file before piping the output

but neither works as expected and I just get .1.1.1.1 files.

Script looks like this (nothing special) and is located on a CentOS box in /etc/cron.hourly:

#!/bin/bash
DATE=`date +%H`
DIR="/abs/path/to/dir"
FILE="hourly-${DATE}.gz"
OPTS="..."
mysqldump $OPTS | gzip -9 > $DIR/$FILE

Can anyone advise as to why this simple operation isn't running as expected?

linux

5 Answers

  1. Best Answer

    Most likely, your script is written to use Bash features, but it's being run by the Bourne shell. Do you have #!/bin/bash as the first line of your script? Please post it so we can better help you.

    Edit:

    In scripts that are meant to run as cron jobs, I always specify the full path to programs (such as mysqldump and gzip) since the $PATH variable and things like aliases are going to be different from those in your interactive shell. That way, the results are predictable.

    • 3

  2. Try adding this line to the top of your script after the bash line

    set +C

    This will turn off the noclobber option in bash so should overwrite the file.

    If it's not that, then it's probably something like an alias to gzip being set either by cron or your overall environment.

    • 2

  3. That seems really strange to me, bash has a noclobber option that makes it so redirection will not overwrite a file. Maybe this a recent version a new option that is similar, but creates a .1 file instead?

    Is it possible something like logrotate is called in the cronjob, or something runs right before it that moves the file (logrotate)? You should check /etc/crontab and crontab -e run as root.

    Is this on some sort of cloud or shared hosting site? Maybe the provider has created something that does this to help reduce their ticket load :-)

    • 1

  4. Try to set #!/bin/bash -x a nd write /path/to/bash_script 1 > /path/to/log_file (where in /path/... set your own path to files).

    • 1

  5. It's not gzip -c? That option is for writing to stdout rather than to a file. Is there a GZIP or GZIP_OPT environment variable set (with gzip options in it)?

    • 0