Home / server / Questions / 209116
In Process
Andrew Fashion
Asked: 2010-12-05 07:11:46 +0800 CST

Delete original tar after untar? Not enough space to untar

  • 772

I have a 130GB tar file I need to untar to /storage.

Problem is, I don't have enough disk space for both to exist at the same time. How can I untar the tar file, and ensure there is enough disk space, and delete the file immediately after untar?

Thanks.

centos tar

5 Answers

  1. This one is hard. It really depends on how much free space you have.

    Others said to extract part of the files and delete them from the tar. That's the only option I can see right now.

    Updating the tar requires the tar to be reconstructed without the deleted files on the same drive. That's why you have to have 2 x tar size + something more to allow extraction of the files.

    HTH

    • 3

  2. If it just a tar file (i.e. not zipped) and if you have enough space left on the disk to store a zipped version of the tarball, you could try to zip this and then construct an unzip | tar pipe. I would test this beforehand, though.

    Easier solution: Get a bigger disk.

    • 1

  3. Can you put the tar file on a different server? Because then, you can untar via ssh without worrying about the space taken by the original file.

    • 1

  4. In a word: Pain and suffering...

    Understanding the file format used by tar, you could build some tools to help with what you are doing, but there are complications that may or may not be relevant to your particular file. The tar file uses headers of 512 bytes that specify the file-name and length of the file, among other things. You could use this information to build up a series of offsets within the tar file of each file entry. You could then do something like traverse the tar file backwards, and truncate the file as you extract the files.

    However, there are some issues of sequence that you have to deal with. GNU tar, for example, can create some "fake" entries for file with long file names, to store additional information that can't fit in the 512 byte header. Also you may need to be careful about directory entries, which might specify permissions that would not allow you to extract files into the directory if you extract the directory entry before the contents.

    The Python programming language, among others, includes a nice library for handling tar files.

    However, another option would be to just split the large tar file up into many smaller files irrespective of the tar format. Split out the end and and then truncate the source file. Repeat until instead of a single 130GB file you have 130 1GB files. Obviously, getting these split/truncates right may be a little tricky. Could be done using the "dd" and "truncate" commands

    Then it would be an easy matter to make a script that would "cat" the first file, delete the first file, cat the second file, delete the second file, etc... Pipe that script to "tar x" and as the tar file extracts the source files would be deleted.

    Of course, these are all destructive operations, so you basically get one shot to do them right.

    The easiest would be if you have a place to copy the 130GB file to and extract it from there. Say an external USB hard drive, or another machine and extract it over an SSH tunnel.

    • 1 
  5. while read p; do
  echo "$p"
  tar -xf archive.tar $p
  tar --delete -f archive.tar $p
done < tar -tf archive.tar
    • 1