I'm trying to build a web client for an HTTPS website. I think I've done everything right, yet obviously I get a different result in the browser than with my crawler.
As far as I understand, there is no way to examine an SSL packet. But, I'm not sure I'm right on this so I'm looking for verification. Presumably SSL traffic goes through some SSL-specific library (openssl) and not everyone re-invents this wheel. Is here anyway I can get wireshark or something to work with it to intercept traffic so I can compare the headers of my client, and Firefox.
Wireshark can decrypt SSL if you have the RSA key for the certificate used by the server. Instructions can be found here.