My website playsongs.pk got 499, 502 and php-fpm.sock failed (11: Resource temporarily unavailable) errors when I received 300-400 active users on site.
Sample/example error logs:
2017-02-27 14:56:23 Error 162.158.78.50 499 GET / HTTP/1.1 0 nginx access
2017-02-27 14:56:23 Error 162.158.78.20 502 GET / HTTP/1.1 166 nginx access
2017-02-27 14:56:23 Error 162.158.78.170 502 GET / HTTP/1.1 166 nginx access
2017-02-27 14:56:23 Error 162.158.79.63 502 GET / HTTP/1.1 166 nginx access
2017-02-27 14:56:23 Error 162.158.78.50 499 GET / HTTP/1.1 0 nginx access
2017-02-27 14:56:23 Error 162.158.78.20 499 GET / HTTP/1.1 0 nginx access
2017-02-27 14:56:23 Error 162.158.78.20 499 GET / HTTP/1.1 0 nginx access
2017-02-27 14:56:23 Error 162.158.78.20 499 GET / HTTP/1.1 0 nginx access
2017-02-27 14:56:23 Error 162.158.78.20 499 GET / HTTP/1.1 0 nginx access
and After few time I got
2017-02-27 14:56:23 Error 172.68.65.4 24026#0: *2784527 connect() to unix:///var/www/vhosts/system/playsongs.pk/php-fpm.sock failed (11: Resource temporarily unavailable) while connecting to upstream nginx error
2017-02-27 14:56:23 Error 172.68.65.112 24023#0: *2784529 connect() to unix:///var/www/vhosts/system/playsongs.pk/php-fpm.sock failed (11: Resource temporarily unavailable) while connecting to upstream nginx error
2017-02-27 14:56:23 Error 172.68.65.100 24028#0: *2784539 connect() to unix:///var/www/vhosts/system/playsongs.pk/php-fpm.sock failed (11: Resource temporarily unavailable) while connecting to upstream nginx error
2017-02-27 14:56:23 Error 172.68.65.112 24021#0: *2784541 connect() to unix:///var/www/vhosts/system/playsongs.pk/php-fpm.sock failed (11: Resource temporarily unavailable) while connecting to upstream nginx error
2017-02-27 14:56:23 Error 172.68.65.112 24021#0: *2784545 connect() to unix:///var/www/vhosts/system/playsongs.pk/php-fpm.sock failed (11: Resource temporarily unavailable) while connecting to upstream nginx error
2017-02-27 14:56:23 Error 172.68.65.112 24021#0: *2784547 connect() to unix:///var/www/vhosts/system/playsongs.pk/php-fpm.sock failed (11: Resource temporarily unavailable) while connecting to upstream nginx error
2017-02-27 14:56:23 Error 172.68.65.112 24028#0: *2784549 connect() to unix:///var/www/vhosts/system/playsongs.pk/php-fpm.sock failed (11: Resource temporarily unavailable) while connecting to upstream nginx error
2017-02-27 14:56:23 Error 162.158.78.50 24028#0: *2784551 connect() to unix:///var/www/vhosts/system/playsongs.pk/php-fpm.sock failed (11: Resource temporarily unavailable) while connecting to upstream nginx error
2017-02-27 14:56:23 Error 172.68.65.112 24028#0: *2784552 connect() to unix:///var/www/vhosts/system/playsongs.pk/php-fpm.sock failed (11: Resource temporarily unavailable) while connecting to upstream nginx error
My Nginx configuration /etc/nginx/nginx.conf:
user nginx;
worker_processes auto; #some last versions calculate it automatically
worker_rlimit_nofile 99999;
error_log /var/log/nginx/error.log crit;
include /etc/nginx/modules.conf.d/*.conf;
events {
worker_connections 66536;
use epoll;
multi_accept on;
}
http {
include mime.types;
default_type application/octet-stream;
open_file_cache max=2048 inactive=20s;
open_file_cache_valid 30s;
open_file_cache_min_uses 5;
open_file_cache_errors off;
access_log off;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 2;
keepalive_requests 100000;
reset_timedout_connection on;
client_body_timeout 30;
send_timeout 15;
client_header_timeout 12;
proxy_connect_timeout 60;
proxy_send_timeout 60;
proxy_read_timeout 60;
fastcgi_buffers 8 128k;
fastcgi_buffer_size 256k;
# reduce the data that needs to be sent over network
gzip on;
gzip_min_length 1000;
gzip_proxied expired no-cache no-store private auth;
gzip_types application/x-javascript text/css application/javascript text/javascript text/plain text/xml application/json application/v$
gzip_disable "MSIE [1-6]\.(?!.*SV1)";
server_tokens off;
include /etc/nginx/conf.d/*.conf;
}
# override global parameters e.g. worker_rlimit_nofile
include /etc/nginx/*global_params;
My php-fpm pool setting are following:
;[php-fpm-pool-settings]
pm = dynamic
pm.max_children = 5000
pm.start_servers = 50
pm.min_spare_servers = 20
pm.max_spare_servers = 70
pm.max_requests = 2000
Please note
Server have 64GB of RAM with 1 GBit dedicate INTERNET PORT and average load on CPU also does not exceed from 1.somthing % or 2.somthing %
Server OS: Centos 7.2
Control Panel: Plesk Onyx
nginx Version: nginx/1.11.4
Output of #cat /proc/$(cat /run/nginx.pid)/limits is following. May it will be helpful to solve the issue.
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 515124 515124 processes
Max open files 1024 4096 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 515124 515124 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
Output of tail /var/log/nginx/error.log
2017/03/01 15:16:15 [notice] 31867#0: getrlimit(RLIMIT_NOFILE): 1024:4096
2017/03/01 15:16:15 [notice] 31868#0: start worker processes
2017/03/01 15:16:15 [notice] 31868#0: start worker process 31869
2017/03/01 15:16:15 [notice] 31868#0: start worker process 31870
2017/03/01 15:16:15 [notice] 31868#0: start worker process 31871
2017/03/01 15:16:15 [notice] 31868#0: start worker process 31872
2017/03/01 15:16:15 [notice] 31868#0: start worker process 31873
2017/03/01 15:16:15 [notice] 31868#0: start worker process 31874
2017/03/01 15:16:15 [notice] 31868#0: start worker process 31875
2017/03/01 15:16:15 [notice] 31868#0: start worker process 31876
Try this:
sysctl net.core.somaxconn=65535php/fpm/pool.d/www.conf:
listen.backlog 65535;Or try using local IP connection instead of unix socket:
php/fpm/pool.d/www.conf:
listen 127.0.0.1:9000;nginx/sites-enabled/sitename:
fastcgi_pass 127.0.0.1:9000;