Ping a Specific Port

Question

claypooj

Asked: 2022-08-21 22:03:17 +0800 CST2022-08-21 22:03:17 +0800 CST 2022-08-21 22:03:17 +0800 CST

Caddy reverse proxy curl works internally, but externally returns content-length: 0

772

Background and Problem

I'm trying to set up Caddy as a reverse proxy between two other web applications and a static file server (all on one machine). When I curl the internal IP, it works as expected, but when I try to curl the external IP, it returns content-length: 0. Ultimately, my question is why is this happening?

I'm not using a domain name, just straight IP. I know I could get a free domain from a bunch of different DNS hosts; I'd really rather not.

Network Setup

I have a router connected to a single single physical server that is home to Caddy and the two web applications. I'm forwarding port 443 from my router to the server where Caddy is installed.

`curl` Results

192.168.1.5 is my internal IP. 203.0.113.0 means my external IP (thanks @Nikita Kipriyanov for pointing out RFC1918 and RFC5737).

It looks like both requests are making it to Caddy at least (based on the HTTP/2 200 line), but I don't understand why when the request comes from the external IP, it returns nothing. Neither request generates any errors in the Caddy terminal output.

Like I mentioned before, I'm not using a domain name, just straight IP. I'm not sure if that has anything to do with the problems I'm seeing, but it seems to me that it shouldn't matter; I've already told curl to ignore the warnings about untrusted certs with the -k option.

Output when using internal IP

admin@server:~$ curl -vk https://192.168.1.5/
*   Trying 192.168.1.5:443...
* TCP_NODELAY set
* Connected to 192.168.1.5 (192.168.1.5) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: [NONE]
*  start date: Aug 21 00:46:28 2022 GMT
*  expire date: Aug 21 12:46:28 2022 GMT
*  issuer: CN=Caddy Local Authority - ECC Intermediate
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x561a91115210)
> GET / HTTP/2
> Host: 192.168.1.5
> user-agent: curl/7.68.0
> accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 200 
< content-type: text/html; charset=utf-8
< cross-origin-opener-policy: same-origin
< referrer-policy: same-origin
< server: Caddy
< x-content-type-options: nosniff
< x-frame-options: DENY
< content-length: 2921
< date: Sun, 21 Aug 2022 05:07:17 GMT
 
<!doctype html>
<html lang="en">
  <body>
    <h1>Hello World!</h1>
  </body>
</html>
* Connection #0 to host 192.168.1.5 left intact

Output when using external IP

admin@server:~$ curl -vk https://203.0.113.0/
*   Trying 203.0.113.0:443...
* TCP_NODELAY set
* Connected to 203.0.113.0 (203.0.113.0) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: [NONE]
*  start date: Aug 21 00:46:28 2022 GMT
*  expire date: Aug 21 12:46:28 2022 GMT
*  issuer: CN=Caddy Local Authority - ECC Intermediate
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55e3ae7aa210)
> GET / HTTP/2
> Host: 203.0.113.0
> user-agent: curl/7.68.0
> accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 200 
< server: Caddy
< content-length: 0
< date: Sun, 21 Aug 2022 05:08:16 GMT
< 
* Connection #0 to host 203.0.113.0 left intact

Caddyfile

192.168.1.5 is my internal IP.

I know app-one has two handles; there's a configuration issue where the app-one drops the trailing / and redirects, which then causes app-two (the one with the catch-all handle) to match, so the fix was to create a second handle just for that case. I'm pretty confident that's not the problem.

{
        default_sni 192.168.1.5
}

https://192.168.1.5:443 {
        handle /file-server/* {
                root * /var/
                file_server browse
        }

        handle /app-one/* {
                reverse_proxy /app-one/* localhost:30000
        }

        handle /app-one {
                reverse_proxy /app-one localhost:30000
        }

        handle {
                reverse_proxy * localhost:8000
        }
}

0 Answers

Voted

claypooj · Answer 1 · 2022-08-24T11:50:42+08:00

The answer from Nikita Kipriyanov was right; I'm just adding this to show what my final configuration looked liked.

Explanation

As Nikita said:

... in the second transcript the Host is different:
GET / HTTP/2
Host: 203.0.113.0
and there is nothing in the Caddyfile to match it. So the behavior is expected.

It turns out the problem was in my Caddyfile. I had improperly configured my site address; Caddy was returning a response with no content because it wasn't matching any routes provided. The final structure of my file looks like this:

{
        default_sni 192.168.1.5
}

https://203.0.113.0:443, https://192.168.1.5:443 {

        handle /static/* {
                root * /srv
                file_server browse
        }

        handle /foundry/* {
                reverse_proxy /foundry/* localhost:30000
        }

        handle /foundry {
                reverse_proxy /foundry localhost:30000
        }

        handle {
                reverse_proxy * localhost:8000
        }
}

A few important things I learned:

The request forwarded from the router does not change the Host header to the internal IP address; it just forwards the whole request to the internal IP address. So instead of receiving a request for the route https://192.168.1.5, Caddy was actually receiving a request for the route https://203.0.113.0. Since there wasn't a route for https://203.0.113.0, it was just sending back a blank page.
The router was configured to listen on port :580 and forward requests to 192.168.1.5:443. I had overlooked this detail in my original question. Something important to note is that the port on the site address to match is not the external port open on the router; it's the port that Caddy is listening to on the machine where Caddy is running. This tripped me up because I'm forwarding port 580 from the router to port 443 on the machine, but in order for Caddy to match the route, I couldn't use https://203.0.113.0:580 (which I had tried as a route during earlier troubleshooting); it has to be https://203.0.113.0:443.

Bottom Line

If you're using IP addresses instead of domain names, make sure your routes for external IPs follow this form:

<http | https>://<external IP>:<port that Caddy is listening on>

Nikita Kipriyanov · Answer 2 · 2022-08-24T03:09:27+08:00

I believe this is due to how address matcher works. Notice that in the first transcript you have (in the HTTP request):

GET / HTTP/2
Host: 192.168.1.5

which is successfully matched with https://192.168.1.5:443 of the Caddyfile, while in the second transcript the Host is different:

GET / HTTP/2
Host: 203.0.113.0

and there is nothing in the Caddyfile to match it. So the behavior is expected.

You can check this by setting arbitrary Host header to 192.168.1.5 with Curl:

curl -H "Host: 192.168.1.5" -vk https://203.0.113.0/

This could work from outside, but there also could be problem with SNI hostname, which will be still set to 203.0.113.0. I am not sure, but adding curl -vk --resolve 192.168.1.5:443:203.0.113.0 https://192.168.1.5/ may help in this case (it is designed to work with host name, like in curl --resolve example.com:443:127.0.0.1 https://example.com/...; I don't know whether it is able to "resolve" IP address to another IP address).

How to fix this? You can try to add other host match to this block:

https://192.168.1.5:443, 203.0.113.0 { ... }

I didn't checked this, but I believe the first one will set the binding, and second is there only to match a Host header.

However, I think better is to never access HTTPS servers by raw IP addresses. Use names.