I have VM1 acting as an Active Directory and DNS server, and have installed SmarterMail (Free) on VM2.
If I remote desktop to VM2 I can connect to the mail server fine if I use 127.0.0.1:25. However if I use the computer name VM2:25 or VM2.domain.local:25 the connection times out.
In DNS I've added an MX record and pointed it back to VM2. This didn't make any difference.
I've checked the firewall settings (although I wouldn't think they would apply when logged on to the local machine) and added a rule for TCP on ports 25 and 110.
Still I can't connect when using the actual machine name. Any ideas?
Update based on Jim B's answer:
I can ping vm2 and get its IPv6 address returned.
Turning off Windows Firewall makes no difference.
I've tried enabling every relevant setting without success. (In terms of relay, the default settings seem to let you log in but won't let you relay without an account.)
I didn't mention there are two NICs on this VM, one set on the internal network of 192.168.200.x and the other set on NAT. I've bound SmarterMail to 192.168.200.x but alas still no progress.
If you cannot connect there is probably a firewall problem. An MX record is used by other mailservers to look up the ip of the mailserver for the domain specified so it will not affect the ability to connect at all.
I would:
ping vm2 and verify the IP returned
turn off windows firewall and see if you can connect to the IP of vm2.
verify that there are no connection setting inside of smartermail that prevent connections from hosts. Some mail servers have settings that will prevent it from taking mail connections from unspecified IP addresses.
I was able to find SmarterMail binding to 127.0.0.1 instead of 192.168.200.2 (the IP of VM2). After changing that I was able to successfully use telnet to connect to ports 25 and 110.
Unfortunately Outlook still refused to connect, but as SmarterMail has a webmail feature, that is a "good enough" solution for now.