The situation
I am running Etherpad, which is proxied via nginx. Etherpad uses Websockets with Socket.io.
My nginx config is more or less this one. The location block for socket.io is this:
rewrite /CustomSubDir/socket.io/(.*) /socket.io/$1 break;
proxy_pass http://localhost:CustomPort/;
proxy_redirect / /CustomSubDir/;
proxy_cookie_path / /CustomSubDir/;
# usual proxy header
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-NginX-Proxy true;
# websocket
proxy_set_header Accept-Encoding "";
proxy_http_version 1.1;
# http://nginx.org/en/docs/http/websocket.html
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header Host $host;
proxy_buffers 8 32k;
proxy_buffer_size 64k;
proxy_buffering off;
What's confusing
So the good message is: Everything is working! I am therefore not asking how to get anything working.
The bas one is: Although everything works nginx always shows me error messages.
What's wrong
nginx shows me these errors in the error.log
:
2016/05/24 xx:yy:zz [error] 22197#0: *12059 connect() failed (111: Connection refused) while connecting to upstream, client: SOM.IPA.DDR.EES, server: somedomain.example.com, request: "GET /CustomSubDir/socket.io/?EIO=3&transport=polling&t=1464121868147-3&sid=H2GhIY24t2a40etpAACd HTTP/2.0", upstream: "http://[::1]:CustomPort/socket.io/?EIO=3&transport=polling&t=1464121868147-3&sid=H2GhIY24t2a40etpAACd", host: "somedomain.example.com"
2016/05/24 xx:yy:zz [error] 22197#0: *12070 connect() failed (111: Connection refused) while connecting to upstream, client: SOM.IPA.DDR.EES, server: somedomain.example.com, request: "POST /CustomSubDir/socket.io/?EIO=3&transport=polling&t=1464122037998-5&sid=T-pthraR669TF2cKAACe HTTP/2.0", upstream: "http://[::1]:CustomPort/socket.io/?EIO=3&transport=polling&t=1464122037998-5&sid=T-pthraR669TF2cKAACe", host: "somedomain.example.com"
So I could trace this requests down. Here is why: 1. These are obviously Websocket requests. 2. And these are - and that's special - POST requests.
When loading an Etherpad or reconnecting to one after a lost connection only one request is made, which fulfils these requirements. I can clearly se it in the browser and can see it appearing in the nginx error log in real-time. This is the request in my browser:
200 POST https://somedomain.example.com/CustomSubDir/socket.io/?EIO=3&transport=polling&t=1464121868143-2&sid=H2GhIY24t2a40etpAACd
And it contains (e.g.) this payload:
164:42["message",{"component":"pad","type":"CLIENT_READY","padId":"CENSORED","sessionID":"null","password":null,"token":"t.qbszmj[...]","protocolVersion":2}]
The server reply is:
HTTP/2.0 200 OK
Server: nginx
Date: Tue, 24 May 2016 xx:yy:zz GMT
Content-Type: text/html
Content-Length: 2
access-control-allow-origin: *
Set-Cookie: io=H2GhIY24t[...]
X-Firefox-Spdy: h2
ok
Why I can be sure this is the culprit
I am very sure the POST request causes this. Not only because it is the only POST request with this URL when accessing the pad, I can also compare the behaviour.
Because on the same server I also run Etherdraw, which works very similar, but it has one important difference: It does not seem to use POST Websocket requests.
And guess what? The error.log
is empty.
My questions
So what are my questions:
- How I can see the request to be successful in my browser (with a correct server reply!) while nginx reports me it is failing in the log?
- How can I get rid of these error messages? AFAIK the requests do not fail...
I could solve my problem thanks to @webzwo0i, who made me aware on GitHub that this might be an IPv4/IPv6 conflict.
So I again looked into the error logs and I noticed especially this:
This is the IPv6 localhost address, but Etherpad/NodeJS only seems to connect to the IPv4 address.
So changing all
localhost
s in the nginx config to127.0.0.1
solved my problem. The errors in the log are gone. I also noticed that some other requests also caused the same errors in the log, so it is not specific to the requests I described.