Home / server / Questions / 36966
Accepted
Joel
Asked: 2009-07-08 04:22:01 +0800 CST

md5sum repeatedly gives different checksum for same file on same machine

  • 772

I have a very small and quite old hard drive disk, about 32G.
On to this disk I have copied a largish tar file, about 5G.

When I run md5sum to generate a checksum on this file I repeatedly get different results (on the same machine and the same file). This obviously should not happen.

If I repeat the experiment with a much smaller file, as expected the checksum is the same each time. I can only assume that because the large file is spanning most of the disk, and it is an old drive, I am experiencing a lot of read errors on the hard drive - and it needs replacing? Could there be any other good reason for this? Something I can do to fix the problem other than buying a new disk?

Update: sha1sum also produces inconsistent results.

hard-drive linux filesystems hash

8 Answers

  1. The only likely reasons are bad disk or bad RAM, and as the disk is old and you are not experiencing other issues it is less likely to be the RAM.

    It could be a dodgey connector/controller: I once had an IDE->USB adaptor go bad resulting in corrupted transfers even thought the drive was perfectly fine.

    • 10
  2. Best Answer

    It may be worth running fsck (or the equivalent for whatever filesystem you're using) to fix disk errors. But, it could be the last thing this drive does if it's on its last leg.

    • 7

  3. To be on the safe side I suggest you run memtest86+ over night to make sure that it finds no memory errors.

    • 4

  4. The root cause is that somehow your RAM is getting corrupted. It is probably a bad stick, but sometimes the devices like the video card can corrupt main memory.

    Do not fsck until you verify that the memory is safe. Otherwise fsck is likely to corrupt everything.

    Pull out drives, put in a known good system, and FSCK there while you fix this system.

    • 3

  5. Does sha1sum have the same behavior?

    I would expect disk read errors before receiving phantom data that alters the hashing process. What platform are you running on?

    • 1

  6. I once had this same problem as well. It turned out to be the hard drive controller on the motherboard.

    However, in this case it definitely sounds like a dud hard drive. Hard drives tend to hang onto life by marking sectors bad, transparently, in the background. When sectors finally start to go bad at an OS level the drive ends up expiring very quickly.

    • 1

  7. Since data is different each time it is possible that md5sum is able to read less and less after each run. But failed reads should cause md5sum to quit and report input/output error.

    It is very important to backup whatever data you have on this drive and shift to new drive. This drive can fail any time. Do not use this drive for any important work.

    • 0

  8. Is there any process or cron job that is set to automatically add or remove files from this tar file?

    I'd also run a disk check. Make sure, as you said, that there aren't any problems with the disk itself.

    • 0