502 error returned from nginx reverse proxy rewrite on some networks

I am getting connection timed out - 502 when connecting from one Internet connection (wifi 4g connection tried with different devices and browsers) while the server works perfectly on VPN, and different Internet connections (tried on several cell networks).

My nginx config is

location ~ /([a-zA-Z0-9]+)$ {
    rewrite ^/([a-zA-Z0-9]+)$ /?page=$1 break;
    proxy_buffering off;
    proxy_pass https://slickalpha.com:443;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-Proto https;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_redirect off;
}

There is another website on the same server, which is loading fine from this connection. And there is no entry on error log or access log of nginx. Please help.

Update:- So after lot of tests, I came to realize the issue was not because of proxy settings but the ipv6 resolving problem. Ill explain further as an answer.