WSUS after Server Upgrade

If you can see all the servers in WSUS - both the old ones and the new ones, with the right IP addresses and recent check-in times - and the only problem is that the new ones are showing the wrong version of Windows, or that some of the names are wrong, the first thing to try is closing and reopening the WSUS console. That will usually work, but if not, then yes, it is safe to delete them and let them add themselves back in, so long as you verify that they do all reappear in due course. (It shouldn't take more than 24 hours, or you can expedite the process as described below.)

If you can't see all the servers, or if some of them didn't reappear after you deleted them, you should be able to confirm which ones are missing by looking at the IP addresses. Check that the missing servers have the right group policy or registry settings to point them at the WSUS server, and correct the settings if necessary. You can expedite the process of getting a server to check in by running a Windows Update scan and then running wuauclt /reportnow on the command line. If you get an error when scanning then you need to address that problem first.

With Windows Server 2016 in particular, the Windows Update client might not work properly out of the box, in which case you will need to download the latest cumulative update from Microsoft and install it manually. Once Windows has been updated, the problem usually goes away and at that point the server should be able to check into WSUS normally.