Why is my console so slow in 18.04?

I just installed 18.04 on a Dell T310 server, and I'm noticing that the text console is unusably slow when scrolling text.

To add some numbers to this, invoking dmesg immediately after boot over an ssh connection takes about half a second to show all of the output, but doing it on the console it takes about 36 seconds for the same amount of output. (A 72x slowdown)

The console stops for a moment immediately after GRUB, and then appears to switch graphics modes to something higher resolution.

Something that worked in the past was forcing vesafb in GRUB (GRUB_CMDLINE_LINUX="video=vesafb:ywrap,mtrr:3" in /etc/default/grub), but that does nothing here.

This has to be something framebuffer related. In dmesg, I see the following:

[    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-4.15.0-72-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro video=vesafb:ywrap,mtrr:3
[    0.212182] pci 0000:01:03.0: BAR 0: assigned to efifb
[    1.115763] efifb: probing for efifb
[    1.115776] efifb: framebuffer at 0xd9800000, using 1216k, total 1216k
[    1.115779] efifb: mode is 640x480x32, linelength=2560, pages=1
[    1.115782] efifb: scrolling: redraw
[    1.115784] efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0
[    1.142136] fb0: EFI VGA frame buffer device
[    4.121850] fb: switching to mgag200drmfb from EFI VGA
[    4.208453] fbcon: mgadrmfb (fb0) is primary device
[    4.726896] mgag200 0000:01:03.0: fb0: mgadrmfb frame buffer device

At this point, I'm more interested in a usable console than I am having it at high resolution (640x480 is perfectly acceptable).

How do I speed this up?

Edit 1:

Setting nomodeset in GRUB_CMDLINE_LINX in /etc/default/grub helped a little, the mode is now down to 640x480 and isn't unusably slow (about 15 seconds to scroll dmesg now), but it's still doing some kind of fancyness I'd rather completely turn off and just have a text mode console.