As mentioned in http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive
It should be particularly noted that the keepalive directive does not limit the total number of connections to upstream servers that a Nginx worker process can open. The connections parameter should be set to a number small enough to let upstream servers process new incoming connections as well.
I want to understand if a new client comes, why can't they use existing keep-alive connections?
Does Nginx need to create a new connection with upstream everytime a new client comes?
I think what it means is that if more requests come in than the number of available keep-alive connections, then a new one would still be opened up nonetheless.
Additionally, note the preceding sentence to the one you quote from http://nginx.org/r/keepalive:
Note the "each worker process" part. I would imagine that if a connection (with a new client request) comes in to a worker process that is already past its number of keepalive connections with a given server, then a new one would still be opened up nonetheless, even if some other worker processes still have some of their keepalive connections with the same upstream server still idle.
If you need more details, it'll probably be worth digging into
ngx_http_upstream_keepalive_module.c
.