I have an application that is getting blocked by a firewall but not logging failures to any type of log. I've since opened the hole in the firewall, but is there a good way to see what the actual request is so I can figured out why this happened in the first place?
Run a network sniffer like tcpdump, or wireshark. You can capture the entire packet and see everything. You can use tcpdump to build captures to be analyzed with wireshark with a command like this tcpdump -qni eth0 -s 0 -w filename.dmp.
One thing you can do to make the troubleshooting easier in the future is simply run two copies of tcpdump. Set one to use the inside interface and one to use the outside interface. Add filters for a specific host if needed. Watch the traffic as it crosses the computer If it comes in one side and doesn't go out the other when it should have you can pretty much assume the firewall blocked it.
Taking a wireshark(ethereal) or tcpdump on both your server and the remote server will give you a complete picture of the traffic.
The easiest option, in my opinion, is to enable logging on any firewall rules applying to the server making the request that you think could be blocking it, or to take a trace on the firewall itself.
If this is all on non-production equipment and you only have administrative access to your client you could use a hub connected to the far side of the firewall to mirror all of the traffic and take a wireshark/tcpdump trace from there.
I used the command
to determine that there were half open TCP requests coming from my server, but I need a good way to see the contents of the requests. The destination server is a machine that routes web service calls to the appropriate machine, so I need more information still to determine what web service was actually getting called.
If you opened the port on the firewall then close it again and enable logging for that filter / proxy. You should be able to see why the firewall is dropping the packets.
Since you have control of the firewall...
If you want to log packets that get dropped, configure logging rules. Something like:
will probably work for you. Modify as appropriate.
(after writing this I realized... your firewall may not be Linux. Hope it is! :) )