user3393046's questions

I have

• php-fpm 7.0.11 - with ioncube
• nginx 1.10.x
• mariadb 10.1

I'm using the PHP-FPM to static, I never had any problems with this. The High CPU usage occurred randomly, and now it's permanent. I didn't make any changes.

At the time we are talking the CPU usage is at 100% with most of the PHP-FPM processes in R state.

Here is a top output My CPU is 16 threads

top - 08:21:15 up 31 min,  2 users,  load average: 799.18, 782.54, 620.78
Tasks: 1082 total, 799 running, 283 sleeping,   0 stopped,   0 zombie
%Cpu(s): 98.5 us,  1.3 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.2 si,  0.0 

My webserver is getting more than 5000 requests / second but with this amount everything was fine before.

Is very strange because:

• If i replace the ioncubed , with the original source of PHP file, the load is decreasing to 8 , as well as the cpu usage to only 15-20%

• If i KEEP the ioncube, but stop the mysql, the load average and cpu is decreasing too.

• Sometimes when i change the sysctl config, the cpu is decreasing for 10 seconds but then again it goes 100%

I can't explain this, because if it was the ioncube, i would have the problem with the 2) but i don't.

I really have lost my hair trying to find the real cause of it and it makes me crazy because that happened when i didn't change anything.

My sysctl file is as follow:

net.core.wmem_max= 1677721600
net.core.rmem_max= 1677721600
net.ipv4.tcp_rmem= 1024000 8738000 1677721600
net.ipv4.tcp_wmem= 1024000 8738000 1677721600
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_timestamps = 1
net.ipv4.tcp_sack = 1
net.ipv4.tcp_no_metrics_save = 1
net.core.netdev_max_backlog = 8000
net.ipv4.route.flush=1
fs.file-max=265536
net.core.somaxconn = 65535
vm.max_map_count=655300

I have searched the whole website and google and couldnt find what i am looking for.

Currently i know that we can limit requests / ip with for example

limit_req_zone $binary_remote_addr zone=one:30m rate=20r/s;

and we can use in any location we want like

limit_req zone=one burst=10 nodelay;

My question is. How do i limit requests globally for all ips at the same time? In short, i have a PHP-FPM location, that i want no matter what, all requests to not exceed 200 requests / second.

I guess we have to replace the $binary_remote_addr, but i have no idea with what.

Here's my config (Nginx 1.7.x) :

server {

    listen 8000;
    index index.php index.html index.htm;
    root /var/www/;
    server_tokens off;
    chunked_transfer_encoding off;

    location ~ \.php$ {
        try_files $uri =404;
        fastcgi_index index.php;
        fastcgi_pass unix:/var/run/php5-fpm.sock;
        include fastcgi_params;
        fastcgi_keep_conn on;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param SCRIPT_NAME $fastcgi_script_name;
    }

}

And php-fpm config:

[www]
user = www-data
group = www-data
listen = /var/run/php5-fpm.sock
listen.owner = www-data
listen.group = www-data
listen.allowed_clients = 127.0.0.1
pm = ondemand
pm.max_children = 6000;
pm.process_idle_timeout = 5;
chdir = /
security.limit_extensions = .php
php_flag[display_errors] = on
php_flag[display_startup_errors] = on

I have a PHP script from which i send LIVE DATA to the client.. When the client closes the connection, I can see that the PID is being closed after 15-20 seconds. I think this is due to buffer and the way nginx works .

For example if I disable the nginx/fastcgi buffering by setting fastcgi_buffering off the connection of the client is being closed instantly.

However, the nginx buffering is very important for me to reduce any lag from the client if found.

So is there any way that nginx/php can instantly get the event when a client disconnects even when I have the buffering on?

Thank you

I tried whole day to make an nginx vhost config to acomplish my needs but i make it wrong all the time. Here is what i need to do.

I have a server in which i stream Movies & Live Streams. The Movies & Streams are being restreamed from a .php file . Everything is fine with this. These 2 files are stream_live.php and stream_movies.php

I want to use different fastcgi_ settings depending on the type value i will select. For example for livestreams you can select to be in mpegts or hls. I want this as a parameter in PHP File. Now i will give examples in order to understand it better.

I want each time a user requests like this:

http://domain.com/live/username/password/1.live?type=hls

to be redirected in the file stream_live.php with parameteres username,password,id and type (1 = id here). How i can do that?

And for the movies i want a similar one like

http://domain.com/movies/username/password/1.movie

to be redirected in the file stream_movies with parameterers Username/Password/ID (1 = ID)

One small note! As i said above i want to use different settings if you are requesting movies or live streams.

So for movies i want to use

• fastcgi_buffering off;

For Live Streams i want to use

• fastcgi_buffering on; EXCEPT in type=hls which i want it fastcgi_buffering off;

Sorry that i'm requesting the whole nginx config but i tried whole day to make it and i couldn't. :(

Thank you

I have a live video streaming in my server using nginx + php.

the php file reas a live streaming and it directly sends it to the client.

I have only one problem. The problem is that i want each request to be in a new process of php-fpm. In a few words i don't want to have idle timeout for a process but instead i want them to close instant when a request is being closed.

With idle timeout i have huge problems which are hard to explain at the moment but i'm really sure that if i disable the idle timeout everything will be perfect.

Is there any way to do this?

I'm using on demand php-fpm

(Since it contains some coding but also it's concerning nginx configuration i think it would be the best to have it in serverfault).

I have some Video Files in my Server that are stored in a folder inside my filesystem and i am using PHP to read the Video File and send it directly to the users using NginX as WebServer.

Most of the Video Files are live streams which i generate using FFmpeg but i have some movies as well.

Live Streaming Files: There are splitted into segments and with php i read the m3u8 file, i get the *.ts files and i am streaming them using PHP while the FFmpeg is still running in background.

Movie Files: Just one static File

I have some questions regarding the nginx/php configuration.

My NginX config is the following:

server {
    listen 80;
    index index.php index.html index.htm;
    root /var/www;
    server_tokens off;
    chunked_transfer_encoding off;


    location ~ \.php$ {    
        try_files $uri =404;
        fastcgi_index index.php;
        fastcgi_pass unix:/var/run/php5-fpm.sock;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param SCRIPT_NAME $fastcgi_script_name;
    }
}

For each Client that reading Videos Files from my Server i log the connection and i can predict if he is still online or no using the

connection_aborted() function from PHP (In a few words, if php script is still running)

Now the Problem:

NginX has by default fastcgi_buffering on; and that is causing to me a problem when i am serving movies to the clients. When i want to serve Live Streaming Files this is fine since i want some buffers to reduce the chances for a lag to occured while PHP is reading the contents of the live streaming files.

But in movies it just parses the entire movie (even if it is 2gb), directly to buffers and can not predict if the client got the response or no. The php script ends in just a second and then nginx is serving the movie to the client so the connection log i talked about before has been ended within one second.

If i turn fastcgi_buffering off; everything is working as i want but i saw some lags in Live Streaming Serving.

The best would be to have fastcgi_buffering on; in live streams and fastcgi_buffering off; in movies. But i have really no idea how to do that.

I tried ob_implicit_flush( true ); but that doesnt work either with NginX i think. In fact, i can not play with any flush() function etc.

The Streaming PHP File uses the following technique to send a video file to the client

<?php

# $video_file can be either a live stream or movie file.

$bytes = 0;

$stream = fopen( $video_file, "rb" );

while ( ! feof( $stream ) && ClientConnected() )
{
    $response = stream_get_line( $stream, 8192 );
    $bytes += strlen( $response );
    echo $response;
}

fclose( $stream );

/*
    $bytes have been sent
    In movie files the bytes directly goes to the filesize of movie file if fastcgi_buffering is on. 
*/

function ClientConnected()
{
    if ( connection_status() != CONNECTION_NORMAL || connection_aborted() )
    {
        return false;
    }

    return true;
}
?>

My Server has the following Specs

CPU: 6 Cores Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz
RAM: 32 GB

I have a problem with nginx+php-fpm. They are taking too many resources for an unknown reason. Even if i restart the nginx + php-fpm the start up processes will use many resources.

My nginx Config is the following:

user  nginx;
worker_processes  auto;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;

worker_rlimit_nofile 300000;
events {
    worker_connections  6000;
    use epoll;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;


#tcp_nopush     on;

keepalive_timeout  65;

#gzip  on;

include /etc/nginx/conf.d/*.conf;

}

My php-fpm pool config is the following

[www]
user = nginx
group = nginx
listen = /var/run/php5-fpm.sock
listen.owner = nginx
listen.group = nginx
listen.allowed_clients = 127.0.0.1
pm = ondemand
pm.max_children = 1500;
pm.process_idle_timeout = 5;
chdir = /
security.limit_extensions = .php

I'm using on pm.ondemand since my website has to support many concurrent connections at the same time and i was unable to to it with dynamic/static.

I guess this isnt the problem because as i said earlier when i restart nginx+php-fpm at the same time, they are taking too much resources without any request.

Here is the screenshot with the CPU Usage

http://s28.postimg.org/v54q25zod/Untitled.png