Does anyone know if it is possible to configure the behaviour of Windows Firewall when it has determined that a packet is disallowed?
By default, it will silently discard the packet and the source is not notified that this has happened, leading to potentially lengthly wait times at the source before the connection timeout ellapses. For a given TCP connection, I want to send a RST packet to an incoming SYN packet that has been disallowed, rather than silently discarding it.
One of my 2K3 boxes does this, but my XP SP3 machine does not. My XP SP3 laptop also used to do it, but my Win7 laptop does not.
The reason this has come about is that I regularly have to connect to several internal Linux based boxes via telnet (they do not have, nor am I able to install, SSH) and they will attempt to use ident to identify me before presenting a username/password prompt. When I connect from the 2K3 server the prompt is instant because the incoming ident packet is actively blocked, but from my XP machine I have to wait around 20 seconds because the remote host is not notified of the blocking. This is not a security thing - everything is local and in no way exposed to the internet, I have 2 firewalls between these local machines and the WAN.
I have done some considerable Googling on the subject and the closest I have come to finding any references to this behaviour is this technet article, but it seems to directly contradict the fact that I get the desired behaviour on one of my boxes. It also contains no information about how configure this type of behaviour, although reading between the lines I think this is a function of the TCP/IP stack, rather than Windows Firewall itself - it seems that the TCP/IP stack (NAT driver) asks the firewall whether a packet is allowed, then handles the action itself.
Before you ask, no, there is definitely no 3rd party firewall software installed on the 2K3 box - neither was there on my old laptop - it is/was just Windows Firewall. The configuration interface is indentical to that of the XP box I am using to write this question. I am fully aware this will most likely involve playing with registry settings, but I have no idea where to start...
Reason - http://technet.microsoft.com/en-us/library/dd448557%28WS.10%29
Fix - http://msdn.microsoft.com/en-us/library/ff720058%28v=prot.10%29.aspx
If Keys/values don't exist - create it. After changes need to restart windows firewall service.
While you have the option to reject connection attempts with an RST or an ICMP-destination-unreachable message with many packet filters, the Windows Firewall seems not to be configurable at this point. The difference in observed behavior is probably due to a disabled firewall - in this case the TCP/IP stack itself would simply send a RST response if the IDENT port is not open.
As a workaround for your case I would recommend exactly this - do add port 113/tcp to the list of exceptions for Windows Firewall. As long as nothing is listening at port 113, you will see the desired RST response from the stack instead of a a series of SYN timeouts and retransmits.
On your Win2k3 box & XP SP3 laptop, are your running any service that is listening on the local port ident is querying for?
The RST could be send e.g. in response to receiving a packet for a closed socket.