On an Azure VM I edited the Windows Firewall rule "RemoteDesktop-UserMode-In-TCP" so that only our Remote IP address was allowed access. In order to do this I also had to select General > Action > Allow the connection if it is secure. For some reason, despite supplying the correct IP address, the connection dropped and I have been unable to logon since. I have tried the steps in http://www.hanselman.com/blog/CSIReenablingRemoteDesktopWithPowerShellAfterYouveBlockedItWithYourOwnFirewallRule.aspx to no avail.
How can I delete / recreate the default RDP firewall rule from Azure Powershell?
Posting as an answer.
Powershell cmdlet to disable Windows Firewall:
Then all you have to do is to connect and remove the problematic rule.