Fixing fonts antialiasing after upgrading to Ubuntu 18.04? (What rendering engines are at hand?)

I have upgraded from Ubuntu +17.10 to 18.04. After that, Gnome wouldn't start - I got a login loop, until I deleted ~/.config/dconf/user. That way I lost all configuration but I have set it back.

However, I can't get the antialiasing to work. I have tried all 3 options in Gnome Tweaks, but I can still see that some fonts are rendered weirdly - with some kind of colorful halo around them. It only seems to happen in some programs, e.g. Intellij IDEA which I think uses GTK through Java, but not sure at all.

Is there some secondary options I have to set? I remember some settings are duplicated between Gnome and KDE. There are Wayland and X. As per this article, I had X but now I should have Wayland. How do I know which one I have? Could switching to the other help (or break more things? :) )