I found some weird entries in my ELB logs that I can't really explain. They all look like this (parsed into JSON):
{
"backend": "-",
"backend_processing_time": "-1",
"backend_status_code": 0,
"client_ip": "XXX.XXX.XXX.XXX",
"client_port": "52084",
"elb": "ELB1-PROD",
"elb_status_code": 504,
"http_method": "POST",
"http_version": "HTTP/1.1",
"received_bytes": 16051,
"request_processing_time": "-1",
"request_uri": "https://example.com:443/example/",
"response_processing_time": "-1",
"sent_bytes": 0,
"ssl_cipher": "ECDHE-RSA-AES128-GCM-SHA256",
"ssl_protocol": "TLSv1.2",
"timestamp": "2015-10-29T04:41:47.251500Z",
"user_agent": "client/4.0"
}
e.g. "backend_processing_time": "-1"
or "backend_status_code": 0
seem really weird.
Any idea what this could mean? I suspected that the backend servers were overloaded, but we didn't experience a higher load than usual when these errors happened, and the errors have all been triggered by the same client IP.
When an ELB returns a 504 Gateway Timeout it indicates that the load balancer closed a connection because a request did not complete within the idle timeout period (according to the docs).
So, the ELB did not receive a response within the idle timeout limit resulting in the connection being killed and a 504 response being returned to the client. With the knowledge that the backend request never finished in mind, it actually makes sense that the
backend_status_code
is0
and that thebackend_processing_time
is-1
. The ELB can't answer something it doesn't know!