Home / ubuntu / Questions / 1189682
Accepted
DKebler
Asked: 2019-11-18 21:40:58 +0800 CST

how to truncate a disk image file of unused space without corrupting GPT partition table (end pointer)

  • 772

Lots of info on taking a image of a disk and shrinking the root (bionic) filesystem partition and then truncating the image to remove the part that became free space. Like https://softwarebakery.com//shrinking-images-on-linux

So there are basically three steps. Use resize2fs to shrink a filesystem on a partition, then decrease the size of the partition as well. Then finally remove the now unused space in the image file.

It is the third part I am having a problem with. Every time I try dd or truncate all the partitions of the image are removed as though the GPT partition table within the image file was trashed.

Here is the original image

GPT fdisk (gdisk) version 1.0.3

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.
Disk rock64-base.img: 30310400 sectors, 14.5 GiB
Sector size (logical): 512 bytes
Disk identifier (GUID): 159DCEDE-DBEA-4657-96D9-2CE178A96B7E
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 30310366
Partitions will be aligned on 64-sector boundaries
Total free space is 30 sectors (15.0 KiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1              64            8063   3.9 MiB     8300  loader1
   2            8064            8191   64.0 KiB    8300  reserved1
   3            8192           16383   4.0 MiB     8300  reserved2
   4           16384           24575   4.0 MiB     8300  loader2
   5           24576           32767   4.0 MiB     8300  atf
   6           32768          262143   112.0 MiB   0700  boot
   7          262144        30310366   14.3 GiB    8300  root

Then after first two steps

Found valid GPT with protective MBR; using GPT.
Disk test.img: 30310400 sectors, 14.5 GiB
Sector size (logical): 512 bytes
Disk identifier (GUID): 159DCEDE-DBEA-4657-96D9-2CE178A96B7E
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 30310366
Partitions will be aligned on 64-sector boundaries
Total free space is 23555002 sectors (11.2 GiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1              64            8063   3.9 MiB     8300  loader1
   2            8064            8191   64.0 KiB    8300  reserved1
   3            8192           16383   4.0 MiB     8300  reserved2
   4           16384           24575   4.0 MiB     8300  loader2
   5           24576           32767   4.0 MiB     8300  atf
   6           32768          262143   112.0 MiB   0700  boot
   7          262144         6755394   3.1 GiB     8300  primary

you can see the root filesystem partition has been reduced to 3.1G

I can load that image fine. I can put it back on a SD card and boot my device. So steps 1 and 2 are fine.

enter image description here

NOW...

if I complete the process doing a `truncate --size=$[(6755394+1)*512] 'test.img'

and

GPT fdisk (gdisk) version 1.0.3

Warning! Disk size is smaller than the main header indicates! Loading
secondary header from the last sector of the disk! You should use 'v' to
verify disk integrity, and perhaps options on the experts' menu to repair
the disk.
Caution: invalid backup GPT header, but valid main header; regenerating
backup header from main header.

Warning! Error 25 reading partition table for CRC check!
Warning! One or more CRCs don't match. You should repair the disk!

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: damaged

****************************************************************************
Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
verification and recovery are STRONGLY recommended.
****************************************************************************
Disk test2.img: 6755395 sectors, 3.2 GiB
Sector size (logical): 512 bytes
Disk identifier (GUID): 159DCEDE-DBEA-4657-96D9-2CE178A96B7E
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 30310366
Partitions will be aligned on 64-sector boundaries
Total free space is 23555002 sectors (11.2 GiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1              64            8063   3.9 MiB     8300  loader1
   2            8064            8191   64.0 KiB    8300  reserved1
   3            8192           16383   4.0 MiB     8300  reserved2
   4           16384           24575   4.0 MiB     8300  loader2
   5           24576           32767   4.0 MiB     8300  atf
   6           32768          262143   112.0 MiB   0700  boot
   7          262144         6755394   3.1 GiB     8300  primary

enter image description here

Clearly truncate (or dd for that matter) are not playing well with GPT. Running gdisk on this file confirms the gpt is trashed

if I run gdisk on the truncated file it reports a trashed GPT.

So obviously I'm missing something here. Apparently when the end of the disk file changes the GPT gets corrupted or needs to be fixed (mismatch). Even though it is fine/present at the end of step 2. I messed around with gdisk and couldn't repair it. Besides that I wanted a solution I could do with a script and with this GPT it doesn't work.

So because it's a GPT do I need to use something other than truncate or dd or do I have to "fix" the GPT manually after truncating.

Here is what gdisk verify reports

Caution: The CRC for the backup partition table is invalid. This table may
be corrupt. This program will automatically create a new backup partition
table when you save your partitions.

Problem: The secondary header's self-pointer indicates that it doesn't reside
at the end of the disk. If you've added a disk to a RAID array, use the 'e'
option on the experts' menu to adjust the secondary header's and partition
table's locations.

Problem: Disk is too small to hold all the data!
(Disk size is 6755395 sectors, needs to be 30310400 sectors.)
The 'e' option on the experts' menu may fix this problem.

Problem: GPT claims the disk is larger than it is! (Claimed last usable
sector is 30310366, but backup header is at
30310399 and disk size is 6755395 sectors.
The 'e' option on the experts' menu will probably fix this problem

Partition(s) in the protective MBR are too big for the disk! Creating a
fresh protective or hybrid MBR is recommended.
partitioning disk-image gpt gdisk

1 Answers

  1. Best Answer

    GPT includes both a main partition table at the start of the disk and a backup partition table at the end of the disk. (Literally the end of the disk -- the final few sectors of the disk.) Thus, when you truncated the disk image, you removed the backup partition table. What's more, some of the pointers and metadata in the main partition table became invalid, since they were pointing past the end of the (virtual) disk. That's most of what the v command in gdisk was complaining about.

    None of this means that the disk is completely trashed. So long as the main partition table's data is valid, gdisk (and most other GPT partitioning tools) can recover. In gdisk, you would type x to enter the experts' menu, then type e to move the backup partition table data to the new end of the disk, and then type w to write the changes to disk. (I'd recommend typing v again before w in case I've overlooked another problem; also, see below....)

    Note, however, that there is one more problem: Your disk image is 6,755,395 sectors in size, according to gdisk; but the final sector of the last partition is at sector 6,755,394. This doesn't leave enough space for the backup partition table (it consumes 33 sectors, by default). If you can re-do the whole process, you should probably shrink the disk image by a smaller amount. If not, try expanding the disk image by 33*512 (16,896) bytes before re-creating the backup partition table with gdisk.

    • 2