Home / ubuntu / Questions / 1457931
Accepted
Gabriel Staples
Asked: 2023-03-06 08:50:01 +0800 CST

How to get old LUKS-encrypted Ubuntu version back into Grub menu after installing new Ubuntu version in new LUKS partition

  • 772

I had a LUKS-encrypted Ubuntu 20.04 setup. I followed my own instructions here to shrink the LUKS partition and install Ubuntu 22.04, also LUKS-encrypted. Now, my grub boot menu no longer gives me the option to boot my old LUKS-encrypted Ubuntu 20.04 version. Rather, it just boots the new 22.04 one.

How do I add the LUKS-encrypted Ubuntu 20.04 version back in?

Here's my disk, as shown in gparted while logged into the new Ubuntu 22.04 OS.

Description:

  1. /dev/nvme0n1p1 is the 512 MiB EFI partition
  2. /dev/nvme0n1p2 is the ext4 /boot non-encrypted partition for the old Ubuntu 20.04 OS
  3. /dev/nvme0n1p3 is the LUKS-encrypted partition containing a single LVM volume with Ubuntu 20.04 in it (no longer in the grub menu)
  4. /dev/nvme0n1p4 is the ext4 /boot non-encrypted partition for the new Ubuntu 22.04 OS
  5. /dev/nvme0n1p5 is the LUKS-encrypted partition containing a single LVM volume with Ubuntu 22.04 in it (is in the grub menu, and is the OS running right now)

enter image description here

Possibly useful:

  1. For more-experienced people, this answer to a different question may have some clues, but I don't understand it: How to get grub to boot from a newly encrypted partition
  2. How to repair /boot on LUKS encrypted harddrive?
dual-boot

1 Answers

  1. Best Answer

    After a lot of effort and investigation, I figured it out!

    1. How to add other LUKS-encrypted Linux distributions back to your Grub bootloader startup menu

    Quick summary

    # 1. Open your `/etc/default/grub` file.
sudo gedit /etc/default/grub
# Then manually add these lines to the bottom of that file:
# (required)
GRUB_DISABLE_OS_PROBER=false
# (optional)
GRUB_ENABLE_CRYPTODISK=y

# 2. Unlock your LUKS-encrypted partitions which contain other bootable
# operating systems. In my case:
sudo cryptsetup luksOpen /dev/nvme0n1p3 nvme0n1p3_crypt

# 3. Update your Grub bootloader in your `/boot` partition. 
sudo update-grub
# When I run `update-grub`, my output now includes this line:
#
#       Found Ubuntu 20.04.5 LTS (20.04) on /dev/mapper/system-root

# 4. Done. Reboot to see and use the new Grub entries!
reboot

    Stop here if you just wanted the quick answer.

    Details

    1. Edit your /etc/default/grub file to add the line GRUB_DISABLE_OS_PROBER=false in it:

      # 1. Open /etc/default/grub in gedit
sudo gedit /etc/default/grub

# 2. Now manually add this line to the bottom of it. 
GRUB_DISABLE_OS_PROBER=false
# I also recommend this line, but it appears to *not* be necessary:
GRUB_ENABLE_CRYPTODISK=y

# 3. Manually save and close the file

      The GRUB_DISABLE_OS_PROBER=false line enables the Operating System prober, which scans your filesystems for valid operating systems and adds them to the Grub bootloader. This feature is disabled by default, so to enable it we set GRUB_DISABLE_OS_PROBER=false. Here is what the Grub user manual has to say about this: https://www.gnu.org/software/grub/manual/grub/html_node/Simple-configuration.html:

      GRUB_DISABLE_OS_PROBER

      The grub-mkconfig has a feature to use the external os-prober program to discover other operating systems installed on the same machine and generate appropriate menu entries for them. It is disabled by default since automatic and silent execution of os-prober, and creating boot entries based on that data, is a potential attack vector. Set this option to false to enable this feature in the grub-mkconfig command.

      Note to self: though it appears to not be necessary: if the above ever doesn't work by itself, try adding GRUB_ENABLE_CRYPTODISK=y too. From the link above:

      GRUB_ENABLE_CRYPTODISK

      If set to y, grub-mkconfig and grub-install will check for encrypted disks and generate additional commands needed to access them during boot. Note that in this case unattended boot is not possible because GRUB will wait for passphrase to unlock encrypted container.

    2. Unlock all LUKS-encrypted partitions which contain operating systems you'd like to add to your Grub menu.

      In my case, my old Ubuntu 20.04 OS is in my /dev/nvme0n1p3 partition, as explained in my question.

      Note: I like to look at my partitions in the gparted GUI partition editor.

      So, here is how to unlock that partition:

      sudo cryptsetup luksOpen /dev/nvme0n1p3 nvme0n1p3_crypt
# You'll need to enter your `sudo` password, as well as the LUKS encryption
# password. It won't show anything while you type them.

# If you have other LUKS-encrypted partitions, unlock them now too. Ex:
sudo cryptsetup luksOpen /dev/nvme0n1p6 nvme0n1p6_crypt
sudo cryptsetup luksOpen /dev/sda2 sda2_crypt
sudo cryptsetup luksOpen /dev/sda3 sda3_crypt
# etc.

      For additional help, see man cryptsetup. Notice that the luksOpen option is equivalent to open --type luks.

    3. Now that your LUKS-encrypted partitions are unlocked, and you have told Grub to search for other operating systems in your filesystems, update the Grub bootloader:

      sudo update-grub

      In its output, look for lines like this, showing that it is finding and adding operating systems to the Grub menu:

      Found Ubuntu 20.04.5 LTS (20.04) on /dev/mapper/system-root

      Full example command and output for me, with that line above in the output:

      $ sudo update-grub
Sourcing file `/etc/default/grub'
Sourcing file `/etc/default/grub.d/init-select.cfg'
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.19.0-32-generic
Found initrd image: /boot/initrd.img-5.19.0-32-generic
Found linux image: /boot/vmlinuz-5.15.0-60-generic
Found initrd image: /boot/initrd.img-5.15.0-60-generic
Memtest86+ needs a 16-bit boot, that is not available on EFI, exiting
Warning: os-prober will be executed to detect other bootable partitions.
Its output will be used to detect bootable binaries on them and create new boot entries.
Found Ubuntu 20.04.5 LTS (20.04) on /dev/mapper/system-root
Adding boot menu entry for UEFI Firmware Settings ...
done

      To verify the new Grub menu entries, you can manually open the file at /boot/grub/grub.cfg and look for menuentry entries after the comment in that file which says ### BEGIN /etc/grub.d/30_os-prober ###. Do not edit that file directly.

    Troubleshooting, debugging, & extra info.

    1. When running sudo update-grub, if you see these lines in the output:

      Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.

      ...then it means you forgot to add GRUB_DISABLE_OS_PROBER=false to your /etc/default/grub file.

    2. If you run sudo update-grub without having first unlocked all of your LUKS-encrypted partitions containing bootable operating systems, then those operating systems will be removed from your Grub menu (if they were previously present) when you run update-grub (see the next section). If your find yourself in this situation, simply boot into whatever Linux OS that is available in the Grub menu, and follow the steps above to add the other OS's back into the Grub menu.

    3. You can see which partitions are unlocked by looking at the output of sudo fdisk -l and lsblk.

      1. With only my running partition (/dev/nvme0n1p5) unlocked (ie: right after a reboot), here's what my output looks like. Notice that in both cases, only nvme0n1p5_crypt is shown:

        # 1. `sudo fdisk -l | grep -i mapper` output:
$ sudo fdisk -l | grep -i mapper
Disk /dev/mapper/nvme0n1p5_crypt: 870.4 GiB, 934587400192 bytes, 1825366016 sectors

# 2. `lsblk` output:
$ lsblk
NAME                MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINTS
loop0                 7:0    0     4K  1 loop  /snap/bare/5
loop1                 7:1    0  63.3M  1 loop  /snap/core20/1778
loop2                 7:2    0  63.3M  1 loop  /snap/core20/1822
loop3                 7:3    0  72.9M  1 loop  /snap/core22/522
loop4                 7:4    0 240.6M  1 loop  /snap/firefox/2356
loop5                 7:5    0 239.8M  1 loop  /snap/firefox/2391
loop6                 7:6    0 400.8M  1 loop  /snap/gnome-3-38-2004/112
loop7                 7:7    0 346.3M  1 loop  /snap/gnome-3-38-2004/119
loop8                 7:8    0 452.4M  1 loop  /snap/gnome-42-2204/56
loop9                 7:9    0  91.7M  1 loop  /snap/gtk-common-themes/1535
loop10                7:10   0  45.9M  1 loop  /snap/snap-store/582
loop11                7:11   0  45.9M  1 loop  /snap/snap-store/638
loop12                7:12   0  49.8M  1 loop  /snap/snapd/17950
loop13                7:13   0  49.8M  1 loop  /snap/snapd/18357
loop14                7:14   0   304K  1 loop  /snap/snapd-desktop-integration/49
loop15                7:15   0   428K  1 loop  /snap/snapd-desktop-integration/57
nvme1n1             259:0    0   1.8T  0 disk  
nvme0n1             259:1    0 953.9G  0 disk  
├─nvme0n1p1         259:2    0   512M  0 part  /boot/efi
├─nvme0n1p2         259:3    0   1.4G  0 part  
├─nvme0n1p3         259:4    0    80G  0 part  
├─nvme0n1p4         259:5    0   1.5G  0 part  /boot
└─nvme0n1p5         259:6    0 870.4G  0 part  
  └─nvme0n1p5_crypt 253:0    0 870.4G  0 crypt /var/snap/firefox/common/host-hunspell
                                               /

      2. Now, after running sudo cryptsetup luksOpen /dev/nvme0n1p3 nvme0n1p3_crypt to also unlock that LUKS-encrypted partition, here's what I see. Notice that I now see nvme0n1p3_crypt and /dev/mapper/system-root as well, where /dev/mapper/system-root is the Logical Volume (LVM) within the /dev/mapper/nvme0n1p3 LUKS-encrypted partition:

        # 1. `sudo fdisk -l | grep -i mapper` output:
$ sudo fdisk -l | grep -i mapper
Disk /dev/mapper/nvme0n1p5_crypt: 870.4 GiB, 934587400192 bytes, 1825366016 sectors
Disk /dev/mapper/nvme0n1p3_crypt: 80 GiB, 85903540224 bytes, 167780352 sectors
Disk /dev/mapper/system-root: 80 GiB, 85899345920 bytes, 167772160 sectors

# 2. `lsblk` output:
$ lsblk
NAME                MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINTS
loop0                 7:0    0     4K  1 loop  /snap/bare/5
loop1                 7:1    0  63.3M  1 loop  /snap/core20/1778
loop2                 7:2    0  63.3M  1 loop  /snap/core20/1822
loop3                 7:3    0  72.9M  1 loop  /snap/core22/522
loop4                 7:4    0 240.6M  1 loop  /snap/firefox/2356
loop5                 7:5    0 239.8M  1 loop  /snap/firefox/2391
loop6                 7:6    0 400.8M  1 loop  /snap/gnome-3-38-2004/112
loop7                 7:7    0 346.3M  1 loop  /snap/gnome-3-38-2004/119
loop8                 7:8    0 452.4M  1 loop  /snap/gnome-42-2204/56
loop9                 7:9    0  91.7M  1 loop  /snap/gtk-common-themes/1535
loop10                7:10   0  45.9M  1 loop  /snap/snap-store/582
loop11                7:11   0  45.9M  1 loop  /snap/snap-store/638
loop12                7:12   0  49.8M  1 loop  /snap/snapd/17950
loop13                7:13   0  49.8M  1 loop  /snap/snapd/18357
loop14                7:14   0   304K  1 loop  /snap/snapd-desktop-integration/49
loop15                7:15   0   428K  1 loop  /snap/snapd-desktop-integration/57
nvme1n1             259:0    0   1.8T  0 disk  
nvme0n1             259:1    0 953.9G  0 disk  
├─nvme0n1p1         259:2    0   512M  0 part  /boot/efi
├─nvme0n1p2         259:3    0   1.4G  0 part  
├─nvme0n1p3         259:4    0    80G  0 part  
│ └─nvme0n1p3_crypt 253:1    0    80G  0 crypt 
│   └─system-root   253:2    0    80G  0 lvm   
├─nvme0n1p4         259:5    0   1.5G  0 part  /boot
└─nvme0n1p5         259:6    0 870.4G  0 part  
  └─nvme0n1p5_crypt 253:0    0 870.4G  0 crypt /var/snap/firefox/common/host-hunspell
                                               /

    4. Note: for UUID (Universal Unique Identifier) numbers for each partition, run this:

      blkid

    5. Note that 3 of the main disk-related files to be aware of are the following. View them with cat:

      # filesystem table (fstab)
cat /etc/fstab

# Encrypted table
cat /etc/crypttab

# User-editable Grub config file
# (read and used by `sudo update-grub`)
cat /etc/default/grub

# NON-user-editable final grub config file on your `/boot` partition
# (created or updated by `sudo update-grub`)
cat /boot/grub/grub.cfg

    2. How to remove other LUKS-encrypted Linux distributions from your Grub bootloader startup menu

    To remove all but the running operating system from the Grub boot menu, simply reboot into your desired operating system (this locks all other LUKS-encrypted partitions again), and run:

    sudo update-grub

    Since no other LUKS-encrypted partitions are unlocked, they'll be automatically removed from the Grub bootloader. Here's what that command and output looks like for me:

    $ sudo update-grub
Sourcing file `/etc/default/grub'
Sourcing file `/etc/default/grub.d/init-select.cfg'
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.19.0-32-generic
Found initrd image: /boot/initrd.img-5.19.0-32-generic
Found linux image: /boot/vmlinuz-5.15.0-60-generic
Found initrd image: /boot/initrd.img-5.15.0-60-generic
Memtest86+ needs a 16-bit boot, that is not available on EFI, exiting
Warning: os-prober will be executed to detect other bootable partitions.
Its output will be used to detect bootable binaries on them and create new boot entries.
Adding boot menu entry for UEFI Firmware Settings ...
done

    If you have a single, shared /boot partition for all of your operating systems (unliked me), then that's it! Running sudo update-grub in this operating system will have updated that single /boot partition, affecting the Grub bootloader used to boot. Or, if you are in your latest-installed operating system, which updates the /boot partition actually used by your Grub bootloader, that's it!

    But, if you're in any other operating system, it will have updated its own, old /boot partition instead of the latest one actually used to boot. In that case, you probably need to remount the latest /boot partition into the /boot dir of your filesystem and then run sudo update-grub again, or perhaps there is some sort of chroot magic you can run. However, I didn't have time for that, so here's a dirty hack I did:

    Quick summary:

    I simply manually copied the newly-created /boot/grub/grub.cfg file into the correct location on the actual (latest) boot partition, since that /boot path it was created in is on the old (now unused) boot partition.

    Details:

    1. I booted into my /dev/nvme0n1p3 Ubuntu 20.04 OS, which had automatically mounted the old (now wrong/unused) /dev/nvme0n1p2 boot partition into the filesystem at path /boot. I ran sudo update-grub to update that (now wrong/unused) /dev/nvme0n1p2 boot partition at path /boot.

    2. I opened a GUI file browser and double-clicked on the new boot partition at /dev/nvme0n1p4, to automatically mount it into the filesystem at path /media/gabriel/abcd-1234-efgh-abcdefghij. In other words, when I am logged into Ubuntu 20.04 in partition /dev/nvme0n1p3, then path /media/gabriel/abcd-1234-efgh-abcdefghij is the exact same as path /boot when I am logged into the Ubuntu 22.04 OS on partition /dev/nvme0n1p5!

    3. Dirty hack: back up the old grub.cfg file and copy the new one into its place.

    4. Summary of the commands:

      # running on the Ubuntu 20.04 OS: forcibly remove the Ubuntu 22.04 entry
# from the Grub boot menu
#--------------------------------------------

# run withOUT first unlocking the Ubuntu 22.04 LUKS partition, so that the
# Ubuntu 22.04 Grub entry will be removed from the "/boot/grub/grub.cfg"
# file
sudo update-grub  

# manually mount the latest boot dir by double-clicking it in your file
# manager

# back up the old grub.cfg file in the new /boot dir
sudo mv /media/gabriel/abcd-1234-efgh-abcdefghij/grub/grub.cfg \
    /media/gabriel/abcd-1234-efgh-abcdefghij/grub/grub.cfg.bak

# dirty hack: copy over the Ubuntu 20.04-created grub.cfg file to the 
# latest /boot dir mounted at this other place
sudo cp /boot/grub/grub.cfg \
    /media/gabriel/abcd-1234-efgh-abcdefghij/grub/grub.cfg

# now reboot, and all you'll see in the Grub boot menu is the old Ubuntu
# 20.04 entry, *not* the new Ubuntu 22.04 entry
reboot

      Another dirty hack to undo the previous dirty hack:

      To undo this, simply restore the old grub.cfg file which you had previously backed up as grub.cfg.bak:

      sudo mv /media/gabriel/abcd-1234-efgh-abcdefghij/grub/grub.cfg.bak \
    /media/gabriel/abcd-1234-efgh-abcdefghij/grub/grub.cfg

    References:

    1. Mostly my own trial and error, but also:
    2. Google search for grub Warning: os-prober will not be executed to detect other bootable partitions
    3. Where I first learned about adding GRUB_DISABLE_OS_PROBER=false to /etc/default/grub: https://forum.manjaro.org/t/warning-os-prober-will-not-be-executed-to-detect-other-bootable-partitions/57849
    4. man update-grub
    5. ***** https://www.gnu.org/software/grub/manual/grub/html_node/Simple-configuration.html
    6. man 8 grub-mkconfig
    7. man cryptsetup
    8. Why do I get "Warning: os-prober will not be executed to detect other bootable partitions." after running "apt upgrade"?
    9. My other question: Unix & Linux: How to get a dual boot (2 Linux OSs) system working when both are LUKS-encrypted
    10. How to repair /boot on LUKS encrypted harddrive?
    11. Adding debian installed on LUKS partition to boot option
    12. How to get grub to boot from a newly encrypted partition <==
    13. Ubuntu install with mdadm and multiple luks partitions
    14. GRUB Bootloader with root LUKS encryption: Only grub shell
    15. man 5 crypttab, and https://man7.org/linux/man-pages/man5/crypttab.5.html
    • 0