I am using AWS to build a service. For this service I want to use ACM certificates. The backend is running on an EC2 instance with TLS enabled using a self-signed certificate. Since ACM certificates can't be exported I want to put a load balancer in front of the backend that will serve the valid certificate.
My Assumptions:
- ALB/NLB support either a https or a tls target.
- ALB/NLB don't validate the certificate of the target group (https://kevin.burke.dev/kevin/aws-alb-validation-tls-reply/)
- The host header is forwarded from the request to the ALB (the application does need a certain host header to work)
Even with this, I am still running into errors.
My current status:
The EC2 instance is reachable on port 443 and reports a self signed certificate (as expected). The EC2 instance is reachable on port 80 for health check purposes.
I've built an ALB with an https target group with my instance as the target. The ALB is reachable via browser and displays my valid certificate, but responds with a 502.
If instead I use an http target group and request the health check endpoint, I'll get a 200 back. For testing purposes I also use a very lax security group for the instance, as well as for the ALB.
So this is what I am seeing in the ALB logs (every line is pretty much the same apart from the time):
https 2020-06-05T09:39:26.715673Z app/testTLSAlb/660e809a22b5d7d2 10.95.21.6:65521 10.95.16.197:443 -1 -1 -1 502 - 240 293 "GET https://my.domain.com:443/adfs/.well-known/openid-configuration HTTP/1.1" "PostmanRuntime/7.25.0" ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 arn:aws:elasticloadbalancing:eu-central-1:accountid:targetgroup/testhttpsTargetGroup/f1e8ce633e9ab069 "Root=1-5eda12ce-aabdb29ae55998867511a310" "my.domain.com" "arn:aws:acm:eu-central-1:accountid:certificate/d9a964b0-6d96-4eeb-b126-2b87f1350942" 0 2020-06-05T09:39:26.714000Z "forward" "-" "-" "10.95.16.197:443" "-"
The URL is reachable if I connect to the target directly. As you may have seen, this is an ADFS Server and that doesn't seem to support real logging.
I have captured a direct request and a request from the ALB with wireshark. The direct request yields a response, the request from the ALB gets a TCP Reset after a TLS Client Hello:
As far as I could see, ciphers should be compatible (the successful request is using a cipher, that the ALB also offers). The only major differences are the TLS extensions that are sent (left is ALB, right is Postman).
The missing SNI field seems to be the problem, since that is a requirement for ADFS (https://docs.microsoft.com/en-us/windows-server/identity/ad-fs/overview/ad-fs-requirements)
In this case the problem was, that ADFS (the application I wanted to run behind the loadbalancer) only responds to TLS Connections that use SNI. Since the ALB does not do that, the ADFS server replied with a TCP reset. For ADFS the solution was to add a default certificate binding by doing the following:
This allows the server to respond, even if no SNI is present in a request.