I have a Laravel application running in production, and there are a few APIs that used a lot. Something was creating a bottleneck and it used to stall our servers (3 with Load Balancer). After optimising the basics in Laravel, caching config, routes, data and so on, even resolving all n+1 issues, we were still having issues in peak times. Someone suggested we run strace on one of the nginx workers to see what is happening on the system level, so we did, and interesting enough, there are lots of redundant system calls where nginx tries to find files when APIs are called:
Part of trace:
240498 stat("/var/www/html/myProject/current/public/APIRequest/3d4f7518e04e9", 0x7ffc7ee6ff70) = -1 ENOENT (No such file or directory)
240498 stat("/var/www/html/myProject/current/public/APIRequest/3d4f7518e04e9", 0x7ffc7ee6ff70) = -1 ENOENT (No such file or directory)
240498 lstat("/var", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
240498 lstat("/var/www", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
240498 lstat("/var/www/html", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
240498 lstat("/var/www/html/myProject", {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0
240498 lstat("/var/www/html/myProject/current", {st_mode=S_IFLNK|0777, st_size=48, ...}) = 0
240498 readlink("/var/www/html/myProject/current", "/var/www/html/myProject/release"..., 4095) = 48
240498 lstat("/var", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
240498 lstat("/var/www", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
240498 lstat("/var/www/html", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
240498 lstat("/var/www/html/myProject", {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0
240498 lstat("/var/www/html/myProject/releases", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
240498 lstat("/var/www/html/myProject/releases/20201202085755", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
240498 lstat("/var/www/html/myProject/releases/20201202085755/public", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
Now the API is called with the ID 3d4f7518e04e9
in this case, and it tries to loop through the directories to find that file instead. But it is not a file, it is an API. We ran strace for less than 30 secs, and we have 5k such calls which don't make sense to me.
So, are these calls necessary? I don't think so, but tell me if I am wrong. And if I am right, how can I configure my nginx better so that these calls can be "caught in time" and resolved appropriately. Any ideas are welcome. :)
PS: We have tried apache with similar config also, same problem in strace appears.
Edit: Do I simply need some sort of a location directive in my site conf to solve this? I am using the basic nginx conf from the official docs https://laravel.com/docs/8.x/deployment#nginx with a few more additions like:
location ~ \.php$ {
fastcgi_pass unix:/var/run/php/php7.2-fpm.sock;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
include fastcgi_params;
fastcgi_buffer_size 128k;
fastcgi_buffers 256 16k;
fastcgi_busy_buffers_size 256k;
fastcgi_temp_file_write_size 256k;
}
location ~ /\.(?!well-known).* {
deny all;
}
client_body_buffer_size 10K;
client_header_buffer_size 1k;
client_max_body_size 8m;
large_client_header_buffers 4 16k;
client_body_timeout 12;
client_header_timeout 12;
keepalive_timeout 15;
send_timeout 10;
EDIT:
location / {
try_files $uri $uri/ /index.php?$query_string;
}
I do have try files as suggested by Danila Vershinin in an answer
It looks like you are dealing with
try_files
performance issue. You can get rid of unnecessarystat
system calls by eliminatingtry_files
from your configuration.The
try_files
directive provides a nice and simple boilerplate for creating SEO-friendly website.However, the downside of this simplicity comes with an added cost of unnecessary
stat
system calls.Since you know that, for e.g. all
/api/
URLs must be routed through your PHP, there is no need to check any files for existence there and you can route through your Laravel bootstrap file unconditionally, e.g.:Moreover, in general, you want to have NGINX cache information about file/directories existence. This can be achieved through open_file_cache.