Home / ubuntu / Questions / 1525473
Accepted
LouBracken
Asked: 2024-09-04 07:21:47 +0800 CST

Decrypt SSD Aftter Failed 24.04 Upgrade

  • 772

This Sunday I tried to update from 22.0.4 to 24.04 but it did not complete, and now I cannot boot to Ubuntu. I suspect something happened with my encrypted SSD which caused grub to fail, and I would like help decrypting it so I can backup my data and then wipe the drive and start fresh.

When I try to boot to Ubuntu I get a black screen which flashes "System reset" and then reboots and dumps me into the BIOS. No BIOS changes were made before this update issue.

From my days of troubleshooting it seems like Grub is messed up, but I cannot repair it using boot-repair or the manual methods I've seen, again probably because of the encrypted SSD. Despite my best efforts I cannot figure out how to decrypt my drive.

Boot-repair provided the following info which has my config info: https://paste.ubuntu.com/p/hgg5nRTyrG/

============================== Boot Info Summary ===============================

 => No boot loader is installed in the MBR of /dev/nvme0n1.

nvme0n1p1: _____________________________________________________________________

    File system:       vfat
    Boot sector type:  FAT32
    Boot sector info:  No errors found in the Boot Parameter Block.
    Operating System:  
    Boot files:        /grub/grub.cfg /efi/BOOT/fbx64.efi /efi/BOOT/mmx64.efi 
                       /efi/ubuntu/fwupdx64.efi /efi/ubuntu/grubx64.efi 
                       /efi/ubuntu/mmx64.efi /efi/ubuntu/shimx64.efi 
                       /efi/ubuntu/grub.cfg

nvme0n1p2: _____________________________________________________________________

    File system:       
    Boot sector type:  Unknown
    Boot sector info: 

nvme0n1p3: _____________________________________________________________________

    File system:       zfs_member
    Boot sector type:  -
    Boot sector info: 

nvme0n1p4: _____________________________________________________________________

    File system:       zfs_member
    Boot sector type:  -
    Boot sector info: 

sda: ___________________________________________________________________________

    File system:       iso9660
    Boot sector type:  Grub2 (v1.99-2.00)
    Boot sector info:  Grub2 (v1.99-2.00) is installed in the boot sector of 
                       sda and looks at sector 0 of the same hard drive for 
                       core.img, but core.img can not be found at this 
                       location.
    Mounting failed:   mount: /mnt/BootInfo/FD/sda: /dev/sda already mounted or mount point busy.


================================ 0 OS detected =================================


================================ Host/Hardware =================================

CPU architecture: 64-bit
Video: UHD Graphics 620 from Intel Corporation
Live-session OS is Ubuntu 64-bit (Ubuntu 22.04.4 LTS, jammy, x86_64)

===================================== UEFI =====================================

BIOS/UEFI firmware: N23ET88W (1.63 )(1.63) from LENOVO
The firmware is EFI-compatible, and is set in EFI-mode for this live-session.
SecureBoot enabled according to mokutil - Please report this message to [email protected].
BootCurrent: 001C
Timeout: 0 seconds
BootOrder: 0000,001A,0010,0011,0012,0013,0014,001C,0018,0019,001B,001D,001E,001F
Boot0000* ubuntu    HD(1,GPT,3715f401-7aab-446d-9692-55f559b7909a,0x800,0x100000)/File(\EFI\ubuntu\shimx64.efi)
Boot0010  Setup FvFile(721c8b66-426c-4e86-8e99-3457c46ab0b9)
Boot0011  Boot Menu FvFile(126a762d-5758-4fca-8531-201a7f57f850)
Boot0012  Diagnostic Splash Screen  FvFile(a7d8d9a6-6ab0-4aeb-ad9d-163e59a7a380)
Boot0013  Lenovo Diagnostics    FvFile(3f7e615b-0d45-4f80-88dc-26b234958560)
Boot0014  Regulatory Information    FvFile(478c92a0-2622-42b7-a65d-5894169e4d24)
Boot0015  Startup Interrupt Menu    FvFile(f46ee6f4-4785-43a3-923d-7f786c3c8479)
Boot0016  Rescue and Recovery   FvFile(665d3f60-ad3e-4cad-8e26-db46eee9f1b5)
Boot0017  MEBx Hot Key  FvFile(ac6fd56a-3d41-4efd-a1b9-870293811a28)
Boot0018* USB CD    VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,86701296aa5a7848b66cd49dd3ba6a55)
Boot0019  USB FDD   VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,6ff015a28830b543a8b8641009461e49)
Boot001A* NVMe0 VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,001c199932d94c4eae9aa0b6e98eb8a400)
Boot001B* ATA HDD0  VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,91af625956449f41a7b91f4f892ab0f601)
Boot001C* USB HDD   VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,33e821aaaf33bc4789bd419f88c50803)
Boot001D* PCI LAN   VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,78a84aaf2b2afc4ea79cf5cc8f3d3803)
Boot001E  Other CD  VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,aea2090adfde214e8b3a5e471856a35406)
Boot001F  Other HDD VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,91af625956449f41a7b91f4f892ab0f606)
Boot0020* USBR BOOT CDROM   PciRoot(0x0)/Pci(0x14,0x0)/USB(11,1)
Boot0021* USBR BOOT Floppy  PciRoot(0x0)/Pci(0x14,0x0)/USB(11,0)
Boot0022* ATA HDD   VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,91af625956449f41a7b91f4f892ab0f6)
Boot0023* ATAPI CD  VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,aea2090adfde214e8b3a5e471856a354)

a9c517741ac31962d7feb152948ad1ee   nvme0n1p1/BOOT/fbx64.efi
a660182adef313615746a665966d2ccc   nvme0n1p1/BOOT/mmx64.efi
d008780449d7affb8a84d3816b167108   nvme0n1p1/ubuntu/fwupdx64.efi
a1da253696a304dce6b4668b70151c0e   nvme0n1p1/ubuntu/grubx64.efi
a660182adef313615746a665966d2ccc   nvme0n1p1/ubuntu/mmx64.efi
64349b3622c65f495a99dbf6102496e3   nvme0n1p1/ubuntu/shimx64.efi
64349b3622c65f495a99dbf6102496e3   nvme0n1p1/BOOT/BOOTX64.efi

============================= Drive/Partition Info =============================

Disks info: ____________________________________________________________________

nvme0n1 : is-GPT,   no-BIOSboot,    has---ESP,  not-usb,    not-mmc, no-os, no-wind,    2048 sectors * 512 bytes

Partitions info (1/3): _________________________________________________________

nvme0n1p1   : no-os,    64, nopakmgr,   no-docgrub, nogrub, nogrubinstall,  grubenv-ok, noupdategrub,   not-far
nvme0n1p2   : no-os,    64, nopakmgr,   no-docgrub, nogrub, nogrubinstall,  no-grubenv, noupdategrub,   not-far

Partitions info (2/3): _________________________________________________________

nvme0n1p1   : is---ESP, part-has-no-fstab,  no-nt,  no-winload, no-recov-nor-hid,   no-bmgr,    notwinboot, vfat
nvme0n1p2   : isnotESP, part-has-no-fstab,  no-nt,  no-winload, no-recov-nor-hid,   no-bmgr,    notwinboot, 

Partitions info (3/3): _________________________________________________________

nvme0n1p1   : not--sepboot, no---boot,  part-has-no-fstab,  not-sep-usr,    no---usr,   part-has-no-fstab,  no--grub.d, nvme0n1
nvme0n1p2   : maybesepboot, no---boot,  part-has-no-fstab,  not-sep-usr,    no---usr,   part-has-no-fstab,  no--grub.d, nvme0n1

fdisk -l (filtered): ___________________________________________________________

Disk nvme0n1: 476.94 GiB, 512110190592 bytes, 1000215216 sectors
Disk identifier: 81898272-66A9-4FCB-B5B8-49697DEBC5C4
           Start        End   Sectors   Size Type
nvme0n1p1    2048    1050623   1048576   512M EFI System
nvme0n1p2 1050624    5244927   4194304     2G Linux swap
nvme0n1p3 5244928    9439231   4194304     2G Solaris boot
nvme0n1p4 9439232 1000215182 990775951 472.4G Solaris root
Disk sda: 29.8 GiB, 31999393792 bytes, 62498816 sectors
Disk identifier: ABBD44C4-640C-4CEA-ACCB-CA38273F602E
       Start      End  Sectors  Size Type
sda1       64  9788791  9788728  4.7G Microsoft basic data
sda2  9788792  9798859    10068  4.9M EFI System
sda3  9798860  9799459      600  300K Microsoft basic data
sda4  9801728 62498752 52697025 25.1G Linux filesystem
Disk zd0: 500 MiB, 524288000 bytes, 1024000 sectors

parted -lm (filtered): _________________________________________________________

sda:32.0GB:scsi:512:512:gpt:PNY USB 3.0 FD:;
1:32.8kB:5012MB:5012MB::ISO9660:hidden, msftdata;
2:5012MB:5017MB:5155kB::Appended2:boot, esp;
3:5017MB:5017MB:307kB::Gap1:hidden, msftdata;
4:5018MB:32.0GB:27.0GB:ext4::;
nvme0n1:512GB:nvme:512:512:gpt:LENSE30512GMSP34MEAT3TA:;
1:1049kB:538MB:537MB:fat32:EFI System Partition:boot, esp;
2:538MB:2685MB:2147MB:::swap;
3:2685MB:4833MB:2147MB:zfs::;
4:4833MB:512GB:507GB:zfs::;
zd0:524MB:unknown:512:8192:loop:Unknown:;
1:0.00B:524MB:524MB:ext4::;

blkid (filtered): ______________________________________________________________

NAME        FSTYPE     UUID                                 PARTUUID                             LABEL                    PARTLABEL
sda         iso9660    2024-02-20-19-39-27-00                                                    Ubuntu 22.04.4 LTS amd64 
├─sda1      iso9660    2024-02-20-19-39-27-00               abbd44c4-640c-4cea-acca-ca38273f602e Ubuntu 22.04.4 LTS amd64 ISO9660
├─sda2      vfat       927C-33D8                            abbd44c4-640c-4cea-acc9-ca38273f602e ESP                      Appended2
├─sda3                                                      abbd44c4-640c-4cea-acc8-ca38273f602e                          Gap1
└─sda4      ext4       d9734087-da6e-4766-b344-b22b17857125 4464a5ce-9a45-1243-882e-69c063850843 writable                 
sdb                                                                                                                       
zd0         ext4       4321cdc4-71ac-4a5d-92b9-5928294a4279                                      keystore-rpool           
nvme0n1                                                                                                                   
├─nvme0n1p1 vfat       8B11-CA84                            3715f401-7aab-446d-9692-55f559b7909a                          EFI System Partition
├─nvme0n1p2                                                 d5bb0e43-1039-a544-a975-43262ee0fbc7                          
├─nvme0n1p3 zfs_member 13143630004745562961                 12e8c81d-a1e4-9f4d-a190-83eba25a8b86 bpool                    
└─nvme0n1p4 zfs_member 15634434524244966334                 f43ee98f-ff09-054c-9739-44ec38389d16 rpool                    

Mount points (filtered): _______________________________________________________

                                                               Avail Use% Mounted on
bpool/BOOT/ubuntu_hzp2t1                                        1.5G  11% /mnt/boot
/dev/nvme0n1p1                                                487.6M   5% /media/ubuntu/8B11-CA84
/dev/sda1                                                          0 100% /cdrom
efivarfs                                                         61K  57% /sys/firmware/efi/efivars

Mount options (filtered): ______________________________________________________

bpool/BOOT/ubuntu_hzp2t1                                      zfs             rw,nodev,relatime,xattr,posixacl,casesensitive
/dev/nvme0n1p1                                                vfat            rw,nosuid,nodev,relatime,uid=999,gid=999,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,showexec,utf8,flush,errors=remount-ro
/dev/sda1                                                     iso9660         ro,noatime,nojoliet,check=s,map=n,blocksize=2048,iocharset=utf8

====================== nvme0n1p1/grub/grub.cfg (filtered) ======================

Revert system only   gnulinux-${root_dataset}-${kversion}
Ubuntu 24.04.1 LTS   gnulinux-rpool/ROOT/ubuntu_hzp2t1-6.8.0-40-generic
### END /etc/grub.d/30_os-prober ###
UEFI Firmware Settings   uefi-firmware
### END /etc/grub.d/30_uefi-firmware ###

=================== nvme0n1p1/efi/ubuntu/grub.cfg (filtered) ===================

search.fs_uuid 8B11-CA84 root 
set prefix=($root)'/grub'
configfile $prefix/grub.cfg

================= nvme0n1p1: Location of files loaded by Grub ==================

           GiB - GB             File                                 Fragment(s)
            ?? = ??             grub/grub.cfg                                  1

======================== Unknown MBRs/Boot Sectors/etc =========================

Unknown BootLoader on nvme0n1p2




================================ ZFS activation ================================

dpkg-query -W -f=${Version} zfsutils-linux : 2.1.5-1ubuntu6~22.04.2
zpool export -f -a 

zpool import -N -R /mnt rpool 
zpool import -N -R /mnt bpool 
cryptsetup -v open /dev/zvol/rpool/keystore zfskey Command failed with code -1 (wrong or missing parameters).
zfs load-key -a 0 / 1 key(s) successfully loaded
zfs mount  
zfs mount -a 
Successfully activated ZFS.
zpool list after activation
NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
bpool  1.88G   208M  1.67G        -         -     0%    10%  1.00x    ONLINE  /mnt
rpool   472G  60.2G   412G        -         -     8%    12%  1.00x    ONLINE  /mnt

zfs list
NAME                                               USED  AVAIL     REFER  MOUNTPOINT
bpool                                              207M  1.55G       96K  /mnt/boot
bpool/BOOT                                         206M  1.55G       96K  none
bpool/BOOT/ubuntu_hzp2t1                           206M  1.55G      206M  /mnt/boot
rpool                                             60.3G   397G      192K  /mnt/media/tempfs
rpool/ROOT                                        19.6G   397G      192K  none
rpool/ROOT/ubuntu_hzp2t1                          19.6G   397G     8.65G  /mnt
rpool/ROOT/ubuntu_hzp2t1/srv                       192K   397G      192K  /mnt/srv
rpool/ROOT/ubuntu_hzp2t1/usr                      2.55M   397G      192K  /mnt/usr
rpool/ROOT/ubuntu_hzp2t1/usr/local                2.36M   397G     2.36M  /mnt/usr/local
rpool/ROOT/ubuntu_hzp2t1/var                      11.0G   397G      192K  /mnt/var
rpool/ROOT/ubuntu_hzp2t1/var/games                 192K   397G      192K  /mnt/var/games
rpool/ROOT/ubuntu_hzp2t1/var/lib                  9.35G   397G     9.14G  /mnt/var/lib
rpool/ROOT/ubuntu_hzp2t1/var/lib/AccountsService   592K   397G      592K  /mnt/var/lib/AccountsService
rpool/ROOT/ubuntu_hzp2t1/var/lib/NetworkManager    560K   397G      560K  /mnt/var/lib/NetworkManager
rpool/ROOT/ubuntu_hzp2t1/var/lib/apt              95.5M   397G     95.5M  /mnt/var/lib/apt
rpool/ROOT/ubuntu_hzp2t1/var/lib/dpkg              122M   397G      122M  /mnt/var/lib/dpkg
rpool/ROOT/ubuntu_hzp2t1/var/log                  1.59G   397G     1.59G  /mnt/var/log
rpool/ROOT/ubuntu_hzp2t1/var/mail                  192K   397G      192K  /mnt/var/mail
rpool/ROOT/ubuntu_hzp2t1/var/snap                 7.27M   397G     7.27M  /mnt/var/snap
rpool/ROOT/ubuntu_hzp2t1/var/spool                4.03M   397G     4.03M  /mnt/var/spool
rpool/ROOT/ubuntu_hzp2t1/var/www                   192K   397G      192K  /mnt/var/www
rpool/USERDATA                                    40.1G   397G      192K  /mnt
rpool/USERDATA/lou_88a2br                         40.1G   397G     40.1G  /mnt/home/lou
rpool/USERDATA/root_88a2br                        27.8M   397G     27.8M  /mnt/root
rpool/keystore                                     518M   397G      443M  -

=================== findmnt (filtered) after ZFS activation ====================

SOURCE                                                        FSTYPE        SIZE   USED  AVAIL USE% TARGET
/dev/sda1                                                     iso9660       4.7G   4.7G      0 100% /cdrom
/dev/disk/by-label/writable[/install-logs-2024-09-03.0/log]   ext4         24.6G   6.7M  23.3G   0% /var/log
/dev/disk/by-label/writable[/install-logs-2024-09-03.0/crash] ext4         24.6G   6.7M  23.3G   0% /var/crash
/dev/nvme0n1p1                                                vfat          511M  23.3M 487.6M   5% /media/ubuntu/8B11-CA84
bpool/BOOT/ubuntu_hzp2t1                                      zfs           1.7G 205.5M   1.5G  11% /mnt/boot

Suggested repair: ______________________________________________________________

The default repair of the Boot-Repair utility would not act on the boot.

I noticed this while I was looking through it:

cryptsetup -v open /dev/zvol/rpool/keystore zfskey Command failed with code -1 (wrong or missing parameters).

If there is a way to fix my boot problem that would be great as well, but my goal is just to get my files so I can start the laptop over from scratch with the latest version.

Thanks!

grub2

1 Answers

  1. Best Answer

    So you were right, @user68168! I wasn't actually using LUKs and that's where I was always getting hung up.

    I don't know how it happened, but I think ZFS couldn't find the key it needed to decrypt, and/or its path changed during the upgrade and messed the rest up.

    The trick came in using zfs mount bpool/BOOT/ubuntu_hzp2t1

    Before I wasn't including "ubuntu_hzp2t1", only up to .../BOOT or .../ROOT

    The commands I used went like this:

    First I made sure that keystore-rpool was available (/dev/zd0) and mounted (/key/) as it contains the decryption key file(?), which I then specified to zfs (...set keylocation=)

    zpool export -a # "unmounted" all zpools if any had been imported
zpool import -N -R /mnt rpool # importing rpool created the /dev/zd0 device
zpool import -N -R /mnt bpool
mkdir /key
mount /dev/zd0 /key/
zfs set keylocation=file:///key/system.key rpool
zfs load-key rpool
zfs mount rpool/ROOT/ubuntu_hzp2t1
zfs mount bpool/BOOT/ubuntu_hzp2t1
zfs mount -a

    I was then able to get to files which were mounted at /mnt/ (you set the zfs mount point using zfs set mountpoint=) and copy them out so I could format and start over.

    Hope this helps someone in the future, thanks for your input and help!

    • 1