On an RHEL5.4 system I setup a script to backup a drive with dd copy every night by cron. I spit the dd output into a log file and email. It is in /etc/crontab and /var/spool/cron/root when I figured out it wouldn't even run under cron.
The script is supposed to copy /dev/sda to /mnt/backup/sda.img (/mnt/backup is a mounted 250gb external).
When I run it as root at the terminal it works fine, I can see data being written to the disk and sda.img is getting bigger.
However when run as cron, I get the output from dd saying it copied 147gb, but cannot find where it spat that 147gb to - it didn't put it in sda.img. Its not on the filesystem anywhere as there is only 50gb left on it.
Where did it go? And how can I make sure the same thing happens in cron that happens in terminal.
I do stop crond and start it before and after the backup, however I am under the impression that cron kicks the job off, I shut it down, it backs up, starts again and is on its merry way.
Thanks.
EDIT: Sorry, the dd line is
dd if=/dev/sda of=/mnt/backup/sda.img bs=400K
And the cron line is
01 0 * * * 2-6 /root/applog_backup.sh
I can access the files when it works with
mount -o loop,offset=32256 sda.img /mnt/restore
I shut down cron to prevent hourly jobs from modifying the disk during backup. I have also shutdown other services and the production database to minimize disk writing in the important places.
You have your "backup" script being executed by cron... and you shut down cron in the script in order to prevent cron jobs from running during the "backup". You really can't see where is the problem here? Your script shuts down crond, but crond is running your script, so, shutting down crond will close the descriptors connected to your script, which will then die, either with a broken pipe or by a interruption signal from crond itself.
Since the script died, crond won't be restarted anymore. That is what we call "shot yourself on the foot".
Even after restarting crond, it won't have registered that the job completed, since it was shutdown during its execution and/or had to signal its termination. Either crond itself or anacron (depends on what cron scheduler you are using), it will have to run the job again, potentially going into an infinite loop.
Your problem is an excellent example of everything that is wrong with inventing your own "backup" solution if you have no real-life experience with reliability management and disaster recovery. Worse, a lack of knowledge of how the system works.
First, and most important, you do not make a raw disk dump on a live filesystem. Filesystems were invented so that you do not touch the raw disk contents directly. You want to save the files stored in the filesystem, that is what matters for you. So you have to access them through the filesystem, not the raw bytes stored on the disk. If the partition is mounted, there is absolute no guarantee that your data is actually stored on the disk and that the disk will stay in a consistent state during the copy.
Even if you could snapshot the state of the disk in a recoverable manner (like a sudden power failure, which could be quickly recoverable with a journaled filesystem like ext3), that is never true with a hot disk dump. A disk dump takes a long time to complete, there are virtually infinite intermediate states between the beginning and the end of the dump, and the dump will contain a mixture of these states, which is potentially unrecoverable even with a journaled filesystem.
And I still didn't mention everything else that is wrong with raw disk dump backups:
Many people have been there and have a lot of experience to share regarding disaster recovery. Don't try to invent your own backup solution, you will end messing things up. Use proper backup tooks, like dump, tar or rsync. If you need something more robust, use Amanda or Bacula, or one of the other hundreds of solutions ready to use.
Probably not the answer you were expecting, but had to be said.
Please, please, please do not use
ddto take backups of your physical volume, for all of the reasons stated in the comments to your question.At the very least if you want to do disk-to-disk backups use something like
rsync(although even that isn't without its problems), and come back here if you can't get that to work fromcron.I suspect that your cron line is not correct. Please edit your crontab file with the below entries:
1 0 * * 2-6 /root/applog_backup.sh
Do let me know if this resolves your issue.
I would suggest that you take the time, and look into a product called Bacula. It's open source (free!), and does a great job. It's complex to setup, but once you get going, you'll forget to worry about backups ever again.