We have a system that we are migrating from AWS to Google Cloud. The servers are containerized and deployed through Kubernetes.
We have the frontend making api calls to the backend. A lot of the time these api calls succeed.
Intermittently, we see that instead of hitting the backend, the api call fails with a 400 response, with the body content set to the generic Google error:
"400. That’s an error.
Your client has issued a malformed or illegal request. That’s all we know. "
We can't see the request hitting the backend in our logs. It's as if it's become lost inside Google.
Doing exactly the same call from the command line from inside the frontend container, or from our local machines works every time.
My questions are:
i) Has anyone seen something similar and have a workaround for the issue?
ii) How can we debug this?
fyi the request is a simple GET that looks like: "https://example.com/v1/session/12345?frontend_ip=1.2.3.4%2C+5.6.7.8&req_timestamp=2018-04-19+10%3A46"
The error can mean several things. It will help if you can share the request you are sending (without any sensitive data).
Additionally it could be the case that they are DELETE or GET requests with body to a ingress resource (HTTP(S) Load Balancer).
In case of DELETE requests with paylaod, it is working as expected as mentioned in this doc.
In case of GET requests with payload, there is an ongoing feature request to allow them.