I've got a linux box set up with 2 network cards to inspect traffic going through port 80. One card is used to go out to the internet, the other one is hooked up to a networking switch. The point is to be able to inspect all HTTP and HTTPS traffic on devices hooked up to that switch for debugging purposes.
I've written the following rules for iptables:
nat
-A PREROUTING -i eth1 -p tcp -m tcp --dport 80 -j DNAT --to-destination 192.168.2.1:1337
-A PREROUTING -i eth1 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 1337
-A POSTROUTING -s 192.168.2.0/24 -o eth0 -j MASQUERADE
On 192.168.2.1:1337, I've got a transparent http proxy using Charles (http://www.charlesproxy.com/) for recording.
Everything's fine for port 80, but when I add similar rules for port 443 (SSL) pointing to port 1337, I get an error about invalid message through Charles.
I've used SSL proxying on the same computer before with Charles (http://www.charlesproxy.com/documentation/proxying/ssl-proxying/), but have been unsuccessful with doing it transparently for some reason. Some resources I've googled say its not possible - I'm willing to accept that as an answer if someone can explain why.
As a note, I have full access to the described set up including all the clients hooked up to the subnet - so I can accept self-signed certs by Charles. The solution doesn't have to be Charles-specific since in theory, any transparent proxy will do.
Thanks!
Edit: After playing with it a little, I was able to get it working for a specific host. When I modify my iptables to the following (and open 1338 in charles for reverse proxy):
nat
-A PREROUTING -i eth1 -p tcp -m tcp --dport 80 -j DNAT --to-destination 192.168.2.1:1337
-A PREROUTING -i eth1 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 1337
-A PREROUTING -i eth1 -p tcp -m tcp --dport 443 -j DNAT --to-destination 192.168.2.1:1338
-A PREROUTING -i eth1 -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 1338
-A POSTROUTING -s 192.168.2.0/24 -o eth0 -j MASQUERADE
I am able to get a response, but with no destination host. In the reverse proxy, if I just specify that everything from 1338 goes to a specific host that I wanted to hit, it performs the hand shake properly and I can turn on SSL proxying to inspect the communication.
The setup is less than ideal because I don't want to assume everything from 1338 goes to that host - any idea why the destination host is being stripped?
Thanks again
The issues you're seeing are the same that prevent the use of multiple certificates on a single IP address/port (without using Server Name Indication).
In plain HTTP, your transparent proxy can tell which host the client wants to connect to by looking into the
Hostheader.When the HTTPS MITM transparent proxy gets the request, it can't know which host name the client was requesting in the first place. (I'm not even sure it can get the IP address with these rules, which might at least have allowed it to make a guess using a reverse DNS lookup, even though it's unlikely to work in the general case.)
Hostheader in the HTTP message, which can only occur after a successful handshake.As a result, the MITM proxy can't know which certificate to generate before the handshake.
This can work with a non-transparent MITM proxy, because you would at least get the intended host name via the HTTP
CONNECTmethod.Just some basic info about this topic.
There are only a few devices that I know of that can succesfully pull off this action. However they are not really available to the common public. I myself am using a Fortinet Fortigate with SSL Offloading.
What it basically does is; it intercepts the SSL Connection made to the host and decrypts the connection in hardware, then inspects where you want to go and makes a firewalling decision based on that information.
Afterwards it sets up it's own connection to that host to retrieve the data and re-signs the original request to the client using a CA provided by the user. To make this work smoothly it is required to have the CA in the trusted root CA on the client.
These kind of setups are used in organisations to enforce company policies regarding internet usage. Since using an Active Directory it is easy to install your company CA in clients this forms no issue for large organisations.
This is the ONLY way you can do it without creating a manual proxy, since SSL traffic IS encrypted. It basically is a MITM so it is important to have any legal issues covered.
There are a few more suggestions on this other question that you may have seen: transparent SSL proxy myths and facts. And there is this link that explains how exactly to configure Squid to become a transparent SSL proxy. It is not what you are looking for, but it might at least give you an insight on what could be going wrong.
The iptables rules seem okay, but I have no idea if the proxy software you are using is able to do what you are trying to do. The documentation certainly claims this to be the case.
To add to Bruno's solution, I investigated a little and wanted to share how I got yet another less-than-ideal quick fix.
After setting those iptables, I can put a reverse proxy on port 1338 and have it forward to localhost on port 1337. Since port 1337 is a transparent http proxy and the data has been decrypted, it will take the host header and make it the destination host.
The major downside is that I've essentially converted an https connection to http - that doesn't always work with every server (not to mention the security hole I'm exposing from that).
I was working from within the limits of my software. I believe a cleaner solution according to Bruno would be to assume all traffic from 1338 should be decrypted. After decryption, inspect the destination host and then proxy the request using SSL.