Home / server / Questions / 878129
Accepted
elbarna
Asked: 2017-10-13 04:31:54 +0800 CST

Compression on tape set..but at 2.27TB..end of space

  • 772

I have a LTO6 tape inserted

tapeinfo -f /dev/st0
Product Type: Tape Drive
Vendor ID: 'QUANTUM '
Product ID: 'ULTRIUM 6       '
Revision: '4142'
Attached Changer API: No
SerialNumber: 'HU1322VW9U'
MinBlock: 1
MaxBlock: 16777215
SCSI ID: 0
SCSI LUN: 0
Ready: yes
BufferedMode: yes
Medium Type: Not Loaded
Density Code: 0x5a
BlockSize: 0
DataCompEnabled: yes
DataCompCapable: yes
DataDeCompEnabled: yes
CompType: 0x1
DeCompType: 0x1
BOP: yes
Block Position: 0
ActivePartition: 0
EarlyWarningSize: 0
NumPartitions: 0
MaxPartitions: 3

But when backup reach the 2.27TB(tape compressed is 6TB) exit with error as tape is not compressed

2,27TiB 8:39:36 [75,6MiB/s] [                                                                        <=>                             ]
pv: write failed: Spazio esaurito sul device
error writing output file

I use tar for backup on slackware 14.2

tar cMpf - -X /etc/file.exclude  /| openssl enc -e -aes256 -salt -pass file:filepass |(pv -p --timer --rate --bytes > /dev/st0)
backup

3 Answers

  1. In your case it is the file level encryption that is preventing compression.

    Encryption tries to make the data stream look as much as random "noise" as possible. Compression tries to increase the data "density" which has a similar effect of limiting further compression.

    • 17
  2. Best Answer

    Compression assumes it can work. tar files generally can not be compressed (they already are), so yes, you may end up not getting the "average compression ratio". Pure text files may compress a lot more. Compression targets are estimates.

    • 5

  3. A couple of, nowadays common, kinds of files will not compress well (achieving far less compression than the ~2.5:1 target assumed), even if they are in an uncompressed archive:

    • anything that is, at any level, compressed already, using any algorithm. This includes gzipped manual pages and documentation, some formats of app bundle, application plugin, or office document (which are pkzip containers internally), installers for software (these are, at their core, often self extracting archives - and often contain media files as described below).

    • As mentioned, anything encrypted

    • modern image and multimedia formats (anything more high-tech than BMP, uncompressed TIFF variants, or WAV audio). These use domain-specific data reduction methods that still result in data that behaves like if it had already been compressed with a format-agnostic method. Also, if these are embedded in other files (eg a TIFF or JPEG image embedded in a PDF, postscript or office document), they make that file far less compressible than expected in turn.

    In some cases, trying to compress any of these can even yield a net gain in file size.

    • 2