WinRM connection refused but not to FQDN

I have 2 domain joined servers running a failover cluster and I am trying to get CAU running. However, the readiness test fails because of WinRM issues.

Both servers get connection refused when using Test-WSMan Server, but can connect when using the FQDN Test-WSMan Server.domain

Test-WSMan : <f:WSManFault xmlns:f="http://schemas.microsoft.com/wbem/wsman/1/wsmanfault" Code="2150859026"
Machine="Server1.domain"><f:Message>The connection to the specified remote host was refused.
Verify that the WS-Management service is running on the remote host and configured to listen for requests on the
correct port and HTTP URL. </f:Message></f:WSManFault>
At line:1 char:1
+ Test-WSMan Server2
+ ~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (Server2:String) [Test-WSMan], InvalidOperationException
    + FullyQualifiedErrorId : WsManError,Microsoft.WSMan.Management.TestWSManCommand

They are on the same subnet, Windows firewalls are off and both have each other in Trusted Hosts. Any ideas why it fails to the hostname but not FQDN?