I got a Nginx over Nginx configuration on my server.
Server scheme
Nginx A -> Nginx B -> PHP FPM
(If you want to know the reason hidden below, it's because one of the two is in a Docker container.)
Problem description
When a client wants to download a big file, Nginx A receive the request, forward it to Nginx B which send it to PHP FPM.
If I send a SIGQUIT signal on Nginx B, it waits until all the requested file has been transmitted, then it closes gracefully.
During this time, Nginx A successfully retransmits all the packets to the client.
BUT: when Nginx B has sent its last packet and it closes, Nginx A captures the closed connection of Nginx B and closes the connection with the client before transmitting the last packet to it.
Experiments
Information retrieved during my tests:
- On average, less than 8 MB is missing from the file sent to the client (tried with small files, and big files like 1 GB or 4 GB)
- Nginx A log an error
upstream prematurely closed connection while sending to client(it shouldn't, it should finish sending the packets to the client and then close this connection without any problem). - If I plug the client connection directly into Nginx B, and send a
SIGQUIT, the client does receive all its packets before the connection is closed
Configuration
Nginx A configuration file:
server_name mywebsite.io;
access_log /var/log/nginx/mywebsite.io.access.log;
error_log /var/log/nginx/mywebsite.io.error.log error;
include /etc/nginx/conf.d/server/default.conf;
include /etc/nginx/conf.d/server/ssl.conf;
include /etc/nginx/conf.d/server/csp.conf;
location / {
client_max_body_size 10G;
client_body_timeout 43200s; # 12h
proxy_read_timeout 1800s; # 30m
proxy_max_temp_file_size 0;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_pass http://127.0.0.1:8000;
}
Nginx B configuration file:
listen 80;
root /var/www/html/public;
server_name _;
index index.php;
client_max_body_size 10G;
location / {
try_files $uri $uri/ /index.php$is_args$args;
}
location ~ \.php$ {
try_files $uri =404;
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass php:9000;
fastcgi_index index.php;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_path_info;
fastcgi_read_timeout 600s; # 10m
}
I tried to tweak many parameters:
client_max_body_size, client_body_timeout, proxy_read_timeout 1800s, proxy_max_temp_file_size, proxy_buffering, proxy_connect_timeout, proxy_send_timeout, proxy_intercept_errors, proxy_buffers, proxy_busy_buffers_size, proxy_buffer_size, proxy_set_header with empty "Connection"...
3 days lost with that, Thanks if you have an idea to help me ! I want to understand what's going on :/
