JimmyBanks's questions

I use Azure DevOps to release a new VM into a VMSS.

When I release the update, Azure swaps the images while the PHP-FPM process is still booting up. This results in 502 Bad Gateway errors for approximately 30 seconds. The PHP-FPM boot process takes roughly a minute to complete.

What I think is happening is that Azure identifies that Nginx is responding and assumes that the VM is fully booted; however, NGINX throws 502 errors because PHP-FPM hasn't completed it's boot process, and it cannot yet find the PHP-FPM socket.

Azure CLI command that is used to release the image: az vmss update

Is there a method to either:

Add a timeout delay before azure swaps the VM? Check to see if NGINX is throwing a 502 error and delay until it isn't?

I have an Ubuntu 18.04 virtual machine that is started up automatically by an Azure VMSS.

It appears that PHP-FPM is taking up to 60 seconds to startup when the VM is turned on. I was under the impression that PHP-FPM startup should be in the low seconds at most. An issue I am running into is that Azure is swapping VMs, and the first 30 seconds I receive a 502 Bad Gateway error because Nginx is searching for php8.1-fpm.sock but the socket hasn't yet been created since FPM is still booting up. I lose a lot of requests during this transition.

My NGINX settings are as follows:

user = www-data
group = www-data
listen = /run/php/php8.1-fpm.sock
listen.owner = www-data
listen.group = www-data
listen.mode = 0660
pm = dynamic
pm.max_children = 35
pm.start_servers = 25
pm.min_spare_servers = 25
pm.max_spare_servers = 30

PHP-FPM is also enabled:

sudo apt-get update
sudo apt-get install -y --no-install-recommends php8.1
sudo apt-get install -y --no-install-recommends php8.1-fpm
sudo apt-get install -y --no-install-recommends php8.1-cli
sudo apt-get install -y --no-install-recommends php8.1-mysql
[...]
sudo systemctl enable php8.1-fpm

What am I missing here that is causing the slow startup times?

As requested in comments: PHP8.1-fpm.log

[24-Jan-2022 15:35:37] NOTICE: fpm is running, pid 12619
[24-Jan-2022 15:35:37] NOTICE: ready to handle connections
[24-Jan-2022 15:35:37] NOTICE: systemd monitor interval set to 10000ms
[24-Jan-2022 15:35:38] NOTICE: Terminating ...
[24-Jan-2022 15:35:38] NOTICE: exiting, bye-bye!
[24-Jan-2022 15:35:38] NOTICE: fpm is running, pid 12719
[24-Jan-2022 15:35:38] NOTICE: ready to handle connections
[24-Jan-2022 15:35:38] NOTICE: systemd monitor interval set to 10000ms
[24-Jan-2022 15:35:46] NOTICE: Terminating ...
[24-Jan-2022 15:35:46] NOTICE: exiting, bye-bye!
[24-Jan-2022 15:35:46] NOTICE: fpm is running, pid 13226
[24-Jan-2022 15:35:46] NOTICE: ready to handle connections
[24-Jan-2022 15:35:46] NOTICE: systemd monitor interval set to 10000ms
[24-Jan-2022 15:35:53] NOTICE: Terminating ...
[24-Jan-2022 15:35:53] NOTICE: exiting, bye-bye!
[24-Jan-2022 15:35:54] NOTICE: fpm is running, pid 14275
[24-Jan-2022 15:35:54] NOTICE: ready to handle connections
[24-Jan-2022 15:35:54] NOTICE: systemd monitor interval set to 10000ms
[24-Jan-2022 15:36:02] NOTICE: Terminating ...
[24-Jan-2022 15:36:02] NOTICE: exiting, bye-bye!
[24-Jan-2022 15:36:02] NOTICE: fpm is running, pid 15050
[24-Jan-2022 15:36:02] NOTICE: ready to handle connections
[24-Jan-2022 15:36:02] NOTICE: systemd monitor interval set to 10000ms
[24-Jan-2022 15:36:12] NOTICE: Terminating ...
[24-Jan-2022 15:36:12] NOTICE: exiting, bye-bye!
[24-Jan-2022 15:36:12] NOTICE: fpm is running, pid 17053
[24-Jan-2022 15:36:12] NOTICE: ready to handle connections
[24-Jan-2022 15:36:12] NOTICE: systemd monitor interval set to 10000ms
[24-Jan-2022 15:36:19] NOTICE: Terminating ...
[24-Jan-2022 15:36:19] NOTICE: exiting, bye-bye!
[24-Jan-2022 15:36:19] NOTICE: fpm is running, pid 17527
[24-Jan-2022 15:36:19] NOTICE: ready to handle connections
[24-Jan-2022 15:36:19] NOTICE: systemd monitor interval set to 10000ms
[24-Jan-2022 15:37:11] NOTICE: Terminating ...
[24-Jan-2022 15:37:11] NOTICE: exiting, bye-bye!
[24-Jan-2022 15:37:11] NOTICE: fpm is running, pid 20214
[24-Jan-2022 15:37:11] NOTICE: ready to handle connections
[24-Jan-2022 15:37:11] NOTICE: systemd monitor interval set to 10000ms
[24-Jan-2022 15:37:19] NOTICE: Terminating ...
[24-Jan-2022 15:37:19] NOTICE: exiting, bye-bye!
[24-Jan-2022 15:37:19] NOTICE: fpm is running, pid 20683
[24-Jan-2022 15:37:19] NOTICE: ready to handle connections
[24-Jan-2022 15:37:19] NOTICE: systemd monitor interval set to 10000ms
[24-Jan-2022 15:37:25] NOTICE: Terminating ...
[24-Jan-2022 15:37:25] NOTICE: exiting, bye-bye!
[24-Jan-2022 15:37:25] NOTICE: fpm is running, pid 21229
[24-Jan-2022 15:37:25] NOTICE: ready to handle connections
[24-Jan-2022 15:37:25] NOTICE: systemd monitor interval set to 10000ms
[24-Jan-2022 15:37:35] NOTICE: Terminating ...
[24-Jan-2022 15:37:35] NOTICE: exiting, bye-bye!
[24-Jan-2022 15:37:35] NOTICE: fpm is running, pid 21650
[24-Jan-2022 15:37:35] NOTICE: ready to handle connections
[24-Jan-2022 15:37:35] NOTICE: systemd monitor interval set to 10000ms
[24-Jan-2022 15:37:40] NOTICE: Terminating ...
[24-Jan-2022 15:37:40] NOTICE: exiting, bye-bye!
[24-Jan-2022 15:37:40] NOTICE: fpm is running, pid 22117
[24-Jan-2022 15:37:40] NOTICE: ready to handle connections
[24-Jan-2022 15:37:40] NOTICE: systemd monitor interval set to 10000ms
[24-Jan-2022 15:56:00] NOTICE: Terminating ...
[24-Jan-2022 15:56:00] NOTICE: exiting, bye-bye!
[24-Jan-2022 16:16:39] NOTICE: fpm is running, pid 1207
[24-Jan-2022 16:16:40] NOTICE: ready to handle connections
[24-Jan-2022 16:16:40] NOTICE: systemd monitor interval set to 10000ms

and systemctl status php8.1-fpm:

 php8.1-fpm.service - The PHP 8.1 FastCGI Process Manager
   Loaded: loaded (/lib/systemd/system/php8.1-fpm.service; enabled; vendor prese
   Active: active (running) since Mon 2022-01-24 16:16:41 UTC; 4h 42min ago
     Docs: man:php-fpm8.1(8)
  Process: 1846 ExecStartPost=/usr/lib/php/php-fpm-socket-helper install /run/ph
 Main PID: 1207 (php-fpm8.1)
   Status: "Processes active: 0, idle: 26, Requests: 123, slow: 0, Traffic: 0req
    Tasks: 131 (limit: 4915)
   CGroup: /system.slice/php8.1-fpm.service
           ├─1207 php-fpm: master process (/etc/php/8.1/fpm/php-fpm.conf)
           ├─1821 php-fpm: pool www
           ├─1822 php-fpm: pool www
           ├─1823 php-fpm: pool www
           ├─1824 php-fpm: pool www
           ├─1825 php-fpm: pool www
           ├─1826 php-fpm: pool www
           ├─1827 php-fpm: pool www
           ├─1828 php-fpm: pool www
           ├─1829 php-fpm: pool www
           ├─1830 php-fpm: pool www
           ├─1831 php-fpm: pool www
           ├─1832 php-fpm: pool www
           ├─1833 php-fpm: pool www

MySQL 8.0.27 has a bug when used to connect to Azure, where it uses all CPU resources and never connects.

Therefore I would like to downgrade to 8.0.26

What I have tried so far:

#uninstall any mysql references
get remove --purge -s 'mysql*'
sudo apt-get autoremove
sudo apt-get autoclean

#install mysql-client 8.0.26
wget -c https://dev.mysql.com/get/mysql-community-client_8.0.26-1ubuntu20.04_amd64.deb
sudo dpkg -i mysql-client_8.0.26-1ubuntu20.04_amd64.deb
sudo apt-get update
sudo apt-get install -y --no-install-recommends mysql-client

This throws the following exception:

mysql-client depends on mysql-community-client (>= 8.0.26-1ubuntu20.04); however:
Package mysql-community-client is not installed.

I assumed that Ubuntu would install the community-client dependency automatically, evidently it did not. How do I go about installing the appropriate dependencies for 8.0.26?

I am using identical configs on Ubuntu 20.04 as I was previously on 18.04; however, on 20.04 the CPU shoots up to 50% usage and the process runs indefinitely, and by indefinitely I mean it was running for over 24hr when I caught the issue. I am completely stumped on why this is not working on 20.04. Mysqldump is installed using the following command:

sudo apt-get update
sudo apt-get install -y --no-install-recommends mysql-client

Output of mysqldump -V Ubuntu 20.04:

mysqldump  Ver 8.0.27-0ubuntu0.20.04.1 for Linux on x86_64 ((Ubuntu))

Output of mysqldump -V Ubuntu 18.04:

mysqldump  Ver 10.13 Distrib 5.7.36, for Linux (x86_64)

Terminal command:

sudo mysqldump --defaults-file=/var/mysqldump/conf.cfg test_db | gzip > /var/tmp/backup.gz

Contents of conf.cfg:

[client]
host=xxx
user=xxx@db
password=xxx

[mysqldump]
quick
single-transaction
skip-lock-tables

Why would this work on Ubuntu 18.04, but fails indefinitely while consuming CPU resources on 20.04? Is there a settings change on 20.04 that I have overlooked?

I would like to direct multiple (but specific) subdomains in a single server block to a path of the same name, using proxy_pass:

api.example.com to www.example.com/api webhook.example.com to www.example.com/webhook iframe.example.com to www.example.com.webook

This directs all requests from api, webhook, and iframe to /api, but I would like to direct api to /api, webhook to /webhook etc.

server {
    listen 443 ssl http2;

    #tls/ssl certificates
    ssl_certificate /etc/ssl/example.crt;
    ssl_certificate_key /etc/ssl/example.key;

    #specify subdomains
    server_name api.example.com webhook.example.com iframe.example.com;

    #reverse proxy to localhost
    location / {
        proxy_pass https://127.0.0.1/api$request_uri;
        proxy_set_header Host "www.example.com";
    }
}

Is it possible to specify the primary directory as a function of the subdomain, or does this need to be three separate server blocks?

I am trying to install mysqldump on a Azure hosted Ubuntu VM.

During the build process I attempt the following command:

sudo apt-get update
sudo apt-get install -y --no-install-recommends mysql-client-8.0

This results in the following error:

Unable to locate package mysql-client-8.0

When I look at the package on packages.ubuntu.com, it is listed under Ubuntu Focal 20.04LTS (https://packages.ubuntu.com/focal/mysql-client-8.0)

If I change the installation command to:

sudo apt-get update
sudo apt-get install -y --no-install-recommends mysql-client

It installed mysql-client-5.7 which isn't running properly. Is the mysql-client version for the appropriate database tied to the Ubuntu server? Should the 5.7 client work correctly with MySQL 8.0?

I am trying to debug the nginx proxy cache and to do so I need to see the $upstream_cache_status value. My config is as follows:

http {
    ...
    proxy_cache_path /tmp/cache_nginx levels=1:2 keys_zone=cfcache:10m max_size=2g inactive=10m use_temp_path=off;
    ...

    server {
        listen 80;
        listen [::]:80;
        server_name domain.com;
        root  /home/site/wwwroot;
        error_log /home/logfiles/nginx/error.log;

        proxy cache cfcache;

        add_header Custom-header-test Value;
        add_header X-Cache-Status $upstream_cache_status always;


        #index file redirect
        index index.php;

        location = /favicon.ico {
          log_not_found off;
          access_log off;
        }

        location = /robots.txt {
          allow all;
          log_not_found off;
          access_log off;
        }

        # Block access to "hidden" files and directories whose names begin with a
        # period. This includes directories used by version control systems such
        # as Subversion or Git to store control files.
        location ~ (^|/)\. {
           return 403;
        }

        location / {
            try_files $uri $uri/ $uri.html @php;
        }

        location @php {         
            rewrite ^(/[^/]+)$ $1.php last;
            rewrite ^(/[^/]+)/(.*)$ $1.php?q=$2 last;
        }

        #404 error page
        error_page 404 /notfound.php;

        location ~ \.php$  {
          try_files $uri =404;
          include fastcgi_params;
          fastcgi_pass unix:/run/php/php7.0-fpm.sock;
          fastcgi_split_path_info ^(.+?\.php)(/.*)?$;
          fastcgi_connect_timeout         300; 
          fastcgi_send_timeout           3600; 
          fastcgi_read_timeout           3600;
          fastcgi_buffer_size 128k;
          fastcgi_buffers 4 256k;
          fastcgi_busy_buffers_size 256k;
          fastcgi_temp_file_write_size 256k;
          fastcgi_intercept_errors on; 
          fastcgi_index index.php;
          fastcgi_param SCRIPT_FILENAME /home/site/wwwroot$fastcgi_script_name;
        }

        #cache static files
        location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|svg+xml)$ {
          expires max;
          log_not_found off;
        }
    }
}

Custom-header-test is appearing as expected, with the value Value

X-Cache-Status on the other hand is not appearing on any requests, why is that? How can I get it to appear?