This is a pre-emptive report (since I am going to post a solution below) for people facing this error when trying to install the recently released Ubuntu 21.04 in BIOS mode (UEFI mode should be fine).
When doing so, and if you happened to have created your media through Rufus (or you installed your GRUB bootloader yourself from the official GRUB 2.04 release) you may find that you get the message:
error: symbol 'grub_register_command_lockdown' not found
Press any key to continue...
and that pressing a key will return you to the GRUB selection screen.