I tried several USB Flash drives and ubuntu-18.04.5-desktop-amd64.iso, ubuntu-mate-20.04.2.0-desktop-amd64.iso and ubuntu-21.04-desktop-amd64.iso with the same result - boot is very slow and live session ends with a full screen wallpaper with no UI components. Mouse right-click opens a popup menu, so I can get to the terminal or Nautilus.
I have i5-9600K CPU, 1TB SSD PC with Ubuntu 18.04.5, which runs just fine. The only unusual thing I noticed is Gparted showing a warning about my boot partition /dev/nvme0n1p1, but only when booted from USB Flash. Warning is about some missing drivers or software components. Any clue what is happening here?
NOTE
Disregard the slow boot. Since all I see is just the wallpaper, I don't really know how long it takes for the OS to be ready.
As @Nmath noticed, the culprit was a second display. In a maze of cables, there was one leading to a display, far, far away. It was rarely used, and I was sure that it was powered down when not in use. Unfortunately, it was on permanent stand by. OS detected it as a primary display and that's why I was getting only a wallpaper on my monitor.