Home / server / Questions / 968175
In Process
cherouvim
Asked: 2019-05-21 23:55:08 +0800 CST

How to disable keepalive in curl

  • 772

I'm trying to figure out how to consistently turn off keepalive across various client machines which are issuing HTTP requests via curl.

This is my target server:

- Ubuntu 18.04.2 LTS
- 4.15.0-47-generic
- HA-Proxy version 1.8.19-1ppa1~bionic 2019/02/12

This is client 1 where from I'm issuing curl (vanilla installation):

- Ubuntu 16.04.3 LTS
- 4.4.0-62-generic
- curl 7.47.0 (x86_64-pc-linux-gnu) libcurl/7.47.0 GnuTLS/3.4.10 zlib/1.2.8 libidn/1.32 librtmp/2.3

This is client 2 where from I'm issuing curl (vanilla installation):

- Ubuntu 18.04 LTS
- 4.15.0-20-generic
- curl 7.58.0 (x86_64-pc-linux-gnu) libcurl/7.58.0 OpenSSL/1.1.0g zlib/1.2.11 libidn2/2.0.4 libpsl/0.19.1 (+libidn2/2.0.4) nghttp2/1.30.0 librtmp/2.3

To turn off keepalive I've tried using -H "Connection: close", --no-keepalive and --keepalive-time 1 and only the first option seems to work but only from client 1.

On client 1 (Ubuntu 16) the connection is not left open but from client 2 (Ubuntu 18) the connection is left open until it times out. I confirm that either by looking at the target server's watch -n 0.1 "netstat -na | fgrep CLIENT_IP_ADDRESS" or by using -vvv on both clients, which on client 1 is always * Closing connection 0 and on client 2 is always * Connection #0 to host www.example.com left intact.

The difference between client 1 and 2 are obviously the Ubuntu version and curl version. What is it that causes the the connection to be closed in client 1 but not in client 2?

I've also found out that if I change my target server to another machine running an old distro with an old apache httpd, whether I send out Connection: close or not doesn't make any difference and keepalive is always used from either of my 2 clients. So I suppose that the target server configuration also plays some role in that.

Edit: To complicate things even further, if from client 1 and 2 I issue requests to the target server via ab then the connection is killed instantly. That is expected because ab uses HTTP/1.0 and not HTTP/1.1. But then, if I target the old distro with apache then in both cases the connection remains open. Which means that, indeed, the receiving end plays a role as well.

Edit 2: I grabbed all /proc/sys/net/ipv4/ settings of both clients via:

for file in /proc/sys/net/ipv4/*
do
  echo "$file $(cat $file)"
done

and these can be found here:

networking

3 Answers

  1. The --no-keepalive option is only useful for TCP keepalive packets, as mentioned in this archived thread on the curl website: https://curl.haxx.se/mail/archive-2013-04/0037.html

    Sounds like you'll need to disable HTTP keepalive packets specifically, which would be done on your servers using keepalive_timeout 0; as mentioned in this stackoverflow thread https://stackoverflow.com/questions/24924237/linux-curlnginx-cant-request-with-no-keepalive.

    Sorry if the formatting is off, this is my first post :)

    Hope that helps!

    • 5

  2. Per https://everything.curl.dev/usingcurl/persist

    curl will always try to keep connections alive and reuse existing connections as far as it can.

    Therefore, you cannot use it using the tools as they are.

    As @pl-nowlan mentioned in their answer, the only way to do that would be on the server side.

    • 3

  3. you can use option --no-keepalive. See more at man curl

    --no-keepalive Disables the use of keepalive messages on the TCP connection, as by default curl enables them.

    Note that this is the negated option name documented. You can thus use --keepalive to enforce keepalive.

    • -3