Danack's questions

I have a server setup with the name 'api.mydomain.com'. I have an error being generated by the application, and the error text is being appropriately logged as:

[error] 9#9: *6916 FastCGI sent in stderr: "application error removed" while reading response header from upstream, client: 172.24.0.5, server: *.api.mydomain.com, request: "GET / HTTP/1.1", upstream: "fastcgi://172.24.0.3:9000", host: "hostmaster.anotherdomain.com"

However...I am completely confused as to where the domain "hostmaster.anotherdomain.com" is coming from.

It is another domain I control, and that domain has a wildcard subdomain setup, so it resolves to the same IP address as 'api.mydomain.com'. But I would much rather than correct host name was used.

Any ideas where that host name can be coming from? Or how to tell (presumably) Nginx to use the correct host name in the log?

I've setup SSL configuration on my webserver through the letsencrypt system.

However it seems that some clients are trying to connect that either have too old versions of SSL, or are deliberately trying to use versions that Letsencrypt won't allow, so I get errors like this in the Nginx log:

2019/08/25 00:51:03 [crit] 30332#30332: *59285 SSL_do_handshake() failed 
(SSL: error:1417D18C:SSL routines:tls_process_client_hello:version
too low) while SSL handshaking, client: 1.2.3.4, server: 0.0.0.0:443

I'm never going to do anything about these events, so I don't need that info to be logged, and they get in the way of viewing actual errors.

Is there any way of telling nginx to not log these types of errors?

We have a PHP server that is running in a container. The logging for PHP-FPM is done by writing to stderr and then kubernetes is forwarding those messages to the appropriate place.

We have some error log messages that have a \n newline character in them.

The result of that is that the error log message is split by new line, with each part being considered a separate log entry, and are also presented in reverse order.

Is there any sensible way of configuring PHP + kuberenetes to have log messages make it through as single entries?

We have a system that we are migrating from AWS to Google Cloud. The servers are containerized and deployed through Kubernetes.

We have the frontend making api calls to the backend. A lot of the time these api calls succeed.

Intermittently, we see that instead of hitting the backend, the api call fails with a 400 response, with the body content set to the generic Google error:

"400. That’s an error.

Your client has issued a malformed or illegal request. That’s all we know. "

We can't see the request hitting the backend in our logs. It's as if it's become lost inside Google.

Doing exactly the same call from the command line from inside the frontend container, or from our local machines works every time.

My questions are:

i) Has anyone seen something similar and have a workaround for the issue?

ii) How can we debug this?

fyi the request is a simple GET that looks like: "https://example.com/v1/session/12345?frontend_ip=1.2.3.4%2C+5.6.7.8&req_timestamp=2018-04-19+10%3A46"

I would like to be able to regenerate the value held by the environment variables inside my testing environment, and have the new values be read by PHP-FPM.

I have written a script that regenerates the environment variables, writes them to a file, and restarts PHP-FPM

# make errors error
set -eufx -o pipefail

# Generate the env settings file
php bin/cli.php genEnvSettings dev /etc/profile.d/imagickdemos.sh

# reload the env values
set +u
source ~/.bashrc
source ~/.bash_profile
set -u

#restart PHP-FPM
/etc/init.d/php-fpm restart

Running this script does not change the environment variables that PHP-FPM receives.

If I run the following commands directly from the CLI after running the above script:

source ~/.bash_profile
/etc/init.d/php-fpm restart

Then the new values for the environment variables are available to PHP-FPM. i.e. it's as if source ~/.bash_profile inside the script has no effect.

How can I reload the environment variables inside that script, rather than having to run the reload directly from the command line? Or is there a better way to make a newly written env settings file be picked up when restarting a service?

I am using Centos 6.4 in case that makes any difference.

I've started creating RPMs of software that I use, to allow me to be able to deploy it configured exactly how I want. However I can't figure out how to rename the package without also renaming the tarball that RPM builds it from.

For example I have the source of PHP5.6 in a tarball php-5.6.0RC3.tar.gz. During the RPM build process this gets extracted to the directory php-5.6.0RC3 and the build process cd's into that directory to make it.

I want to change the package name to phpcustom, and still have it built from php-5.6.0RC3.tar.gz. The extract process works, but then the build process tries to cd into the directory phpcustom-5.6.0RC3 which obviously doesn't exist, and so it breaks.

How do I tell the rpmbuild which directory the build should take place in?

I guess I could just move the php-5.6.0RC3 directory to be phpcustom-5.6.0RC3 ...but it seems a bit silly.

I've put my spec file below, just in case it is relevant.

%define        __spec_install_post %{nil}
%define          debug_package %{nil}
%define        __os_install_post %{_dbpath}/brp-compress

Summary: Custom built PHP with APCU
Name: php
#Name: phpcustom - this doesn't work
Provides: php
Conflicts: php
Version: 5.6.0RC3
Release: 1
License: None
Group: Development/Tools

Requires: bzip2, libcurl, libxml2

SOURCE0 : http://downloads.php.net/tyrael/php-5.6.0RC3.tar.gz
SOURCE1:        php.ini
SOURCE2:        php-cli.ini
SOURCE3:        apcu-4.0.6.tgz
URL: http://php.net/


%description
%{summary}

%prep
%setup -q -n php-%{version}
%setup -T -D -a 3

%build
mkdir -p  %{buildroot}
./configure  \
                --disable-cgi \
                --disable-debug \
                --disable-rpath \
                --disable-xmlreader \
                --disable-xmlwriter \
                --disable-xml \
                --enable-fpm \
                --enable-intl \
                --enable-json \
                --enable-mbregex \
                --enable-mbstring \
                --enable-pcntl \
                --enable-pdo \
                --enable-sockets \
                --enable-sysvsem \
                --enable-sysvshm \
                --enable-zip \
                --with-apcu \
                --with-bz2 \
                --with-config-file-path=/etc \
                --with-curl \
                --with-freetype-dir=/usr/lib \
                --with-gd \
                --with-jpeg-dir=/usr/lib \
                --with-mcrypt \
                --with-png-dir=/usr/lib \
                --with-pdo-mysql \
                --with-zlib \
                --with-mhash \
                --with-mysql \
                --with-mysqli=mysqlnd \
                --with-openssl \
                --with-pcre-regex \
                --without-pear \
                --without-zlib \
                --enable-maintainer-zts
make -j4


%install
rm -rf %{buildroot}
mkdir -p %{buildroot}%{_initrddir}
install -Dp -m0755 sapi/fpm/init.d.php-fpm.in %{buildroot}%{_initrddir}/php-fpm
%{__make} install INSTALL_ROOT="%{buildroot}"
cp %{SOURCE1} %{buildroot}/etc/php.ini
cp %{SOURCE2} %{buildroot}/etc/php-cli.ini

%post
%/sbin/chkconfig --add php-fpm
%/sbin/chkconfig --level 2345 php-fpm on


%clean
 rm -rf %{buildroot}


%preun
if [ "$1" = 0 ] ; then
    /sbin/service php-fpm stop > /dev/null 2>&1
    /sbin/chkconfig --del php-fpm
fi
exit 0

%postun
if [ "$1" -ge 1 ]; then
    /sbin/service php-fpm condrestart > /dev/null 2>&1
fi
exit 0



%files
%defattr(-,root,root,-)
%config(noreplace) %{_sysconfdir}/php.ini
%config(noreplace) %{_sysconfdir}/php-cli.ini
/etc/rc.d/init.d/php-fpm
/usr/local/bin/*
/usr/local/etc/*
/usr/local/include/php/*
/usr/local/lib/php/build/*
/usr/local/php/fpm/*
/usr/local/php/man/man1/*
/usr/local/php/man/man8/*
/usr/local/sbin/*
/usr/local/lib/php/extensions/no-debug-zts-20131226/opcache.a
/usr/local/lib/php/extensions/no-debug-zts-20131226/opcache.so


%changelog
* Thu Apr 24 2009  Elia Pinto <[email protected]> 1.0-1
- First Build

EOF

I am running PHP as PHP-FPM behind Nginx. Occasionally one of the pools stops responding to requests. It seems that nginx is accepting the connection, passing it to PHP-FPM but then PHP-FPM is not giving a response. This happens for all requests to that pool, including those that should normally run very quickly.

I have logging enabled for slow PHP-FPM. Normally this gives a stack trace of what functions had been called when the slow log time was reached, but in the slow log I see:

[08-Jun-2014 16:40:49]  [pool imagick] pid 16825
script_filename = /home/github/imagick-demos//imagick/index.php
[0x00007fa70269c1c8] +++ dump failed

and in the PHP-FPM log I see:

WARNING: [pool imagick] child 17502, script '/home/github/imagick-demos//imagick/index.php' (request: "GET /index.php") executing too slow (10.459610 sec), logging
NOTICE: child 17502 stopped for tracing
NOTICE: about to trace 17502
ERROR: failed to ptrace(PEEKDATA) pid 17502: Input/output error (5)
NOTICE: finished trace of 17502

i.e. the trace of what is causing the slowness has failed.

The other PHP-FPM pools are responding fine. What's weird is that just restarting PHP-FPM does not appear to fix this. However when it just happened now, I did the following:

1. Restarted PHP-FPM multiple times to no effect.
2. Disabled Xdebug and OPCache, and restarted PHP-FPM.
3. Re-enabled Xdebug and OPCache, and restarted PHP-FPM.

After that the pool started responding again normally. I was monitoring PHP-FPM with strace, and there is nothing obviously blocking or failing in there.

What can cause PHP-FPM to be blocked like this?

I'm compiling PHP from source. I would like to set the location that PHP-FPM looks for it's config file.

Apparently this option should be set via the configure option:

--with-fpm-conf=/etc/php/php-fpm.conf

According to http://php-fpm.org/wiki/Configuration_File

However using this option on PHP5.5.0 gives the error configure: WARNING: unrecognized options: --with-fpm-conf

How do I set where PHP-FPM should look for it's conf file when compiling from source?

I have my Nginx server setup allow requests to be served with the X-Accel-Redirect header. This works fine for images, but I'm trying to get it to work for Javascript files that are already gzip'ed on the disk and it appears Nginx is stripping off the Content-Encoding header. This makes the browser not unzip the content when it receives it, which obviously makes it not work.

How can I serve pre-gzipped content through X-Accel-Redirect ?

The headers seen are:

With Accel = doesn't work

Accept-Ranges:bytes
Cache-Control:max-age=1209600, s-maxage=120960
Connection:keep-alive
Content-Length:122871
Content-Type:application/javascript
Date:Mon, 24 Jun 2013 13:44:36 GMT
Expires:Mon, 08 Jul 2013 13:44:36 GMT
Last-Modified:Mon, 24 Jun 2013 13:38:20 GMT
Server:nginx/1.2.8

No accel = works

Cache-Control:max-age=1209600, s-maxage=120960
Connection:keep-alive
Content-Encoding:gzip
Content-Length:122871
Content-Type:application/javascript
Date:Mon, 24 Jun 2013 13:45:08 GMT
Expires:Mon, 08 Jul 2013 13:45:08 GMT
HTTP1/0 200 Ok:
Pragma:cache
Server:nginx/1.2.8
X-Powered-By:PHP/5.4.9

In case it's relevant the code that is serving the content is:

public static function  proxyFile($fileNameToServe, $mimeType, $alreadyGzip = false){
        $seconds_to_cache = 3600 * 24 * 7 * 2; 

        $filesize = filesize($fileNameToServe);
        header('Content-Length: '.$filesize);

        if ($alreadyGzip) {
            header('Content-Encoding: gzip');
        }

        if(defined('X-ACCEL-REDIRECT') == true && constant('X-ACCEL-REDIRECT') == true){
            $filenameToProxy = str_replace(PATH_TO_ROOT."var/cache", '/protected_files', $fileNameToServe );
            sendProxyHeaders($mimeType, $seconds_to_cache);
            header("X-Accel-Redirect: ".$filenameToProxy);
            exit(0);
        }
        else{
            sendProxyHeaders($mimeType, $seconds_to_cache);
            $fileHandle = fopen($fileNameToServe, 'r');

            if($fileHandle == false){
                throw new \Exception("Failed to open file [$fileNameToServe] for serving.");
            }

            while (!feof($fileHandle)) {
                $contents = fread($fileHandle, 8192);
                echo $contents;
            }

            fclose($fileHandle);
        }

        exit(0);
    }

I've tried setting proxy_pass_header Content-Encoding; but apparently it has no effect.

I have a location block setup to catch all file requests and send them to PHP-FPM:

location  / {
    try_files  $uri /routing.php?$args;
    fastcgi_pass   unix:/opt/local/var/run/php54/php-fpm-www.sock;
    include       /documents/projects/intahwebz/intahwebz/conf/fastcgi.conf;
}

This works and correctly passes the request to PHP-FPM to either the exact existing php file that was requested or with routing.php set as the script to run.

I tried to add an error page, so that if routing file was ever removed or otherwise not available, the error page would be shown rather than Nginx's default error page:

location  / {
    try_files $uri /routing.php?$args /50x_static.html;
    fastcgi_pass   unix:/opt/local/var/run/php54/php-fpm-www.sock;
    include       /documents/projects/intahwebz/intahwebz/conf/fastcgi.conf;
}

This stops the routing.php file from being served, and the 50x_static.html page is shown instead. Requests to an existing PHP file still work i.e. going to the URL /dynamic.php

I realise that the last parameter in a try_files command is slightly magical:

In the event that no file is found, an internal redirect to the last parameter is invoked. Do note that only the last parameter causes an internal redirect, former ones just sets the internal URI pointer. The last parameter is the fallback URI and must exist, or else an internal error will be raised.

While investigating why the error_page had broken the config, I realised that for the config that works (without the static error page), Nginx does appear to be matching the request twice according to the Nginx rewrite log when trying to get the root URL "/":

"^/proxy/(\d+)/(\w+)/(.+)\.(gif|png|jpg|jpeg|GIF|PNG|JPG|JPEG)$" does not match "/", client: 127.0.0.1, server: basereality.com, request: "GET / HTTP/1.1", host: "basereality.test"
"^/proxy/(\d+)/(.+)\.(gif|png|jpg|jpeg|GIF|PNG|JPG|JPEG)$" does not match "/", client: 127.0.0.1, server: basereality.com, request: "GET / HTTP/1.1", host: "basereality.test"
"^/staticImage/(\w+)/(.+)\.([^\.]*)$" does not match "/", client: 127.0.0.1, server: basereality.com, request: "GET / HTTP/1.1", host: "basereality.test"
"^/proxy/(\d+)/(\w+)/(.+)\.(gif|png|jpg|jpeg|GIF|PNG|JPG|JPEG)$" does not match "/routing.php", client: 127.0.0.1, server: basereality.com, request: "GET / HTTP/1.1", host: "basereality.test"
"^/proxy/(\d+)/(.+)\.(gif|png|jpg|jpeg|GIF|PNG|JPG|JPEG)$" does not match "/routing.php", client: 127.0.0.1, server: basereality.com, request: "GET / HTTP/1.1", host: "basereality.test"
"^/staticImage/(\w+)/(.+)\.([^\.]*)$" does not match "/routing.php", client: 127.0.0.1, server: basereality.com, request: "GET / HTTP/1.1", host: "basereality.test"

i.e. the request comes in as /, the try_files fails to serve the file and so rewrites the request from / to /routing.php and then reprocesses the request.

Why is try files not serving the routing.php file on the first pass? It exists and is accessible, otherwise it wouldn't be served on the second time round.

EDIT

Removed unrelated config.

I'm migrating a set of bash scripts that deploy a server to Puppet. Almost all of the functionality has been moved into Puppet, but I can't see how to migrate one bit of the bash script that does the following:

1. Install PHP.
2. With a PHP script, generate a random password and store it as a variable in the bash script.
3. Use that variable elsewhere in the bash script to set the MySQL root password.

How can I set a puppet variable dynamically like this?

btw I'm using Puppet in a standalone version using puppet apply as I don't have a separate server to act as the puppet master. Also, in case anyone is wondering why it's done like this, it's so that the MySQL root password doesn't exist anywhere except on the box where it's used and isn't written to any (readable) files.

Are there any 'boxes' available to download for Vagrant that closely mirror the Linux version and packages available for Amazon EC2 instances?

I am testing locally using Vagrant, currently using the Precise32 box. I am deploying my code to Amazon EC2 instances with the 'Amazon Linux AMI 2013.03' AMI.

The problem I am facing is that the packages available to install on the Precise32 box have different names e.g.

• php5-fpm - in the vagrant box
• php54-fpm - in the Amazon EC2 instance.

Although I could fix this by using variables in the Puppet config files to handle the different package names, that sounds a bit like a hack.

So are there any vagrant boxes available that will mirror the Amazon EC2 environment closely.

Or is it possible for me to tell a vagrant box to use the same packages source as an Amazon EC2 instance?

To clarify one thing - the reason I want to do this is so that I can test locally without having to commit everything first. Although Vagrant has support for AWS as a provider, that wouldn't actually achieve being able to test locally.

I'm using Vagrant to setup a virtual machine for testing. I would like apt inside the virtual machine to use a proxy on the host machine to allow caching of all the downloads to persist between instances of the virtual machine running, not only for improved speed but to allow me to start the vagrant instance when I have no internet connection.

I've setup the proxy which is running, and I thought I'd told apt to use it by setting this in the Vagrant script:

config.vm.provision :shell, :inline => 'echo \'Acquire { Retries "0"; HTTP { Proxy "http://10.0.2.2:3128"; }; };\' >> /etc/apt/apt.conf'
config.vm.provision :shell, :inline => 'echo \'Acquire { Retries "0"; FTP { Proxy "ftp://10.0.2.2:3128"; }; };\' >> /etc/apt/apt.conf'
config.vm.provision :shell, :inline => 'echo \'Acquire { Retries "0"; HTTPS { Proxy "https://10.0.2.2:3128"; }; };\' >> /etc/apt/apt.conf'

And it's working partially i.e. the initial requests for sources hit the cache and work when my wifi connection is disabled but Squid is running and has some entries in it's cache. I can also see some requests hitting Squid from Squids log file:

1367492453.816     34 127.0.0.1 TCP_REFRESH_MODIFIED/200 592 GET http://security.ubuntu.com/ubuntu/dists/precise-security/Release.gpg - DIRECT/91.189.92.181 -
1367492453.987    168 127.0.0.1 TCP_REFRESH_MODIFIED/200 49973 GET http://security.ubuntu.com/ubuntu/dists/precise-security/Release - DIRECT/91.189.92.181 -
1367492453.999    325 127.0.0.1 TCP_MISS/404 588 GET http://us.archive.ubuntu.com/ubuntu/dists/precise/InRelease - DIRECT/91.189.91.13 text/html
1367492454.113    114 127.0.0.1 TCP_MISS/404 596 GET http://us.archive.ubuntu.com/ubuntu/dists/precise-updates/InRelease - DIRECT/91.189.91.13 text/html

However, the bulk of the download of packages are not being cached and doesn't appear to go through Squid at all i.e. I can see large network usage when they're downloaded and they don't appear in the Squid access log.

So my question how do I configure Apt to use a proxy for all it's requests?

btw I also tried configuring by setting the environment variable http_proxy:

 config.vm.provision :shell, :inline => "echo 'export http_proxy=http://10.0.2.2:3128' >> /etc/profile.d/proxy.sh"

This has the same effect - some requests appear to be hitting Squid but not all of them, and particularly not the actual packages.

How can I find out what files or sockets are needed by Curl when running in a chroot'ed PHP environment on Mac OSX?

I recently secured my website by chrooting PHP but now every call to Amazon S3 through it's API is failing, with the error message:

Resource id #43; cURL error: Couldn't resolve host 's3.amazonaws.com' 

The same code works fine when called from a non-chroot`ed command line so presumably it's some set of files that need to be opened by CURL that are failing to be opened.

How do I figure out which files it's trying to open to be able to map them into the chroot'ed directoy?

With Andrei's suggestion I used strace (or rather dtruss the mac equivalent) to get some debug output.

This is a request that is chrooted and doesn't work.

lstat64("/lib/amazonWS/lib\0", 0x7FFF5FBF97D0, 0x26)         = 0 0
lstat64("/lib/amazonWS\0", 0x7FFF5FBF9630, 0x182)        = 0 0
open("/lib/amazonWS/lib/requestcore/requestcore.class.php\0", 0x0, 0x1B6)        = 5 0
fstat64(0x5, 0x100A9F448, 0x8)       = 0 0
fstat64(0x5, 0x100A9F448, 0x90)      = 0 0
fstat64(0x5, 0x100A9F448, 0x1001B1FE4)       = 0 0
mmap(0x0, 0x72D1, 0x1, 0x1, 0x5, 0xFFFFFF8016E89160)         = 0x3BF0000 0
stat64("/lib/amazonWS/lib/requestcore/requestcore.class.php\0", 0x10621CD68, 0x10621C847)        = 0 0
munmap(0x103BF0000, 0x72D1)      = 0 0
close(0x5)       = 0 0
setitimer(0x2, 0x7FFF5FBFBCE0, 0x0)      = 0 0
kqueue(0x7FFF5FBFB000, 0x0, 0x50)        = 5 0
kevent(0x5, 0x7FFF5FBFB000, 0x1)         = 0 0
socket(0x1, 0x1, 0x0)        = 6 0
setsockopt(0x6, 0xFFFF, 0x1022)      = 0 0
connect_nocancel(0x6, 0x7FFF5FBFADD0, 0x6A)      = -1 Err#2
close_nocancel(0x6)      = 0 0
close_nocancel(0x5)      = 0 0
kqueue(0x7FFF5FBFB000, 0x0, 0x50)        = 5 0
kevent(0x5, 0x7FFF5FBFB000, 0x1)         = 0 0
socket(0x1, 0x1, 0x0)        = 6 0
setsockopt(0x6, 0xFFFF, 0x1022)      = 0 0
connect_nocancel(0x6, 0x7FFF5FBFADD0, 0x6A)      = -1 Err#2
close_nocancel(0x6)      = 0 0
close_nocancel(0x5)      = 0 0
open_nocancel(".\0", 0x0, 0x0)       = 5 0
fstat64(0x5, 0x7FFF5FBFBA80, 0x0)        = 0 0
fcntl_nocancel(0x5, 0x32, 0x7FFF5FBFBCB0)        = 0 0
close_nocancel(0x5)      = 0 0
stat64("/basereality\0", 0x7FFF5FBFB9F0, 0x0)        = 0 0
lstat64("/basereality/../BaseReality/html/50x.html\0", 0x7FFF5FBFB730, 0x23D)        = 0 0

This is a request that is not chrooted and works:

 1806/0x1dd3:    149524       5      3 open("/documents/projects/intahwebz/intahwebz/lib/amazonWS/lib/requestcore/requestcore.class.php\0", 0x0, 0x1B6)      = 5 0
 1806/0x1dd3:    149525       3      0 fstat64(0x5, 0x100A9F578, 0x8)        = 0 0
 1806/0x1dd3:    149527       2      0 fstat64(0x5, 0x100A9F578, 0x90)       = 0 0
 1806/0x1dd3:    149528       2      0 fstat64(0x5, 0x100A9F578, 0x1001B1FE4)        = 0 0
 1806/0x1dd3:    149531       5      2 mmap(0x0, 0x72D1, 0x1, 0x1, 0x5, 0x1F)        = 0x3BF0000 0
 1806/0x1dd3:    149535       4      2 stat64("/documents/projects/intahwebz/intahwebz/lib/amazonWS/lib/requestcore/requestcore.class.php\0", 0x1061966D0, 0x106196057)      = 0 0
 1806/0x1dd3:    149565       6      3 munmap(0x103BF0000, 0x72D1)       = 0 0
 1806/0x1dd3:    149568       4      2 close(0x5)        = 0 0
 1806/0x1dd3:    149661       4      1 setitimer(0x2, 0x7FFF5FBFBCE0, 0x0)       = 0 0
 1806/0x1dd3:    149767      90      2 kqueue(0x7FFF5FBFB000, 0x0, 0x50)         = 5 0
 1806/0x1dd3:    149770       4      1 kevent(0x5, 0x7FFF5FBFB000, 0x1)      = 0 0
 1806/0x1dd3:    149782      12      8 socketpair(0x1, 0x1, 0x0)         = 0 0
 1806/0x1dd3:    149792      12      9 sendto_nocancel(0x6, 0x105D8FCF0, 0x3A)       = 58 0
 1806/0x1dd3:    149796       7      3 sendmsg_nocancel(0x6, 0x7FFF5FBFAD30, 0x0)        = 1 0
 1806/0x1dd3:    149797       4      0 close_nocancel(0x9)       = 0 0
 1806/0x1dd3:    149802      45      4 recvfrom_nocancel(0x7, 0x7FFF5FBFAD7C, 0x4)       = 4 0
 1806/0x1dd3:    149806       5      2 close_nocancel(0x7)       = 0 0
 1806/0x1dd3:    149814       6      3 socketpair(0x1, 0x1, 0x0)         = 0 0
 1806/0x1dd3:    149816       4      1 sendto_nocancel(0x6, 0x105D8FCF0, 0x3A)       = 58 0
 1806/0x1dd3:    149818       5      2 sendmsg_nocancel(0x6, 0x7FFF5FBFAD30, 0x0)        = 1 0
 1806/0x1dd3:    149819       3      0 close_nocancel(0x9)       = 0 0
 1806/0x1dd3:    149822     215      2 recvfrom_nocancel(0x7, 0x7FFF5FBFAD7C, 0x4)       = 4 0
 1806/0x1dd3:    149840      20     17 close_nocancel(0x7)       = 0 0
 1806/0x1dd3:    149843       4      1 kevent(0x5, 0x7FFF5FBFB000, 0x1)      = 0 0
 1806/0x1dd3:    149844       3      1 kevent(0x5, 0x0, 0x0)         = 1 0
 1806/0x1dd3:    149846       2      0 kevent(0x5, 0x0, 0x0)         = 1 0
 1806/0x1dd3:    149848       5      2 recvfrom_nocancel(0x6, 0x7FFF5FBFAE30, 0x1C)      = 28 0
 1806/0x1dd3:    149851       5      1 recvfrom_nocancel(0x6, 0x105D8FCF0, 0x40)         = 64 0
 1806/0x1dd3:    149856       8      3 select_nocancel(0x7, 0x7FFF5FBFADB0, 0x0)         = 1 0
 1806/0x1dd3:    149861       3      1 kevent(0x5, 0x7FFF5FBFA860, 0x1)      = 0 0
 1806/0x1dd3:    149863       4      1 recvfrom_nocancel(0x6, 0x7FFF5FBFAE30, 0x1C)      = 28 0
 1806/0x1dd3:    149864       3      1 recvfrom_nocancel(0x6, 0x105D85990, 0x42)         = 66 0
 1806/0x1dd3:    149867       4      1 select_nocancel(0x7, 0x7FFF5FBFADB0, 0x0)         = 0 0
 1806/0x1dd3:    149869       2      0 kevent(0x5, 0x7FFF5FBFA860, 0x1)      = 0 0
 1806/0x1dd3:    149871       3      1 kevent(0x5, 0x0, 0x0)         = 1 0
 1806/0x1dd3:    149872       2      0 kevent(0x5, 0x0, 0x0)         = 1 0
 1806/0x1dd3:    149873       2      0 kevent(0x5, 0x0, 0x0)         = 1 0
 1806/0x1dd3:    149874       2      0 kevent(0x5, 0x0, 0x0)         = 1 0
 1806/0x1dd3:    149876       4      1 recvfrom_nocancel(0x6, 0x7FFF5FBFAE30, 0x1C)      = 28 0
 1806/0x1dd3:    149877       4      0 recvfrom_nocancel(0x6, 0x105D859E0, 0x40)         = 64 0
 1806/0x1dd3:    149879       3      1 select_nocancel(0x7, 0x7FFF5FBFADB0, 0x0)         = 1 0
 1806/0x1dd3:    149881       2      0 kevent(0x5, 0x7FFF5FBFA860, 0x1)      = 0 0
 1806/0x1dd3:    149883       3      0 recvfrom_nocancel(0x6, 0x7FFF5FBFAE30, 0x1C)      = 28 0
 1806/0x1dd3:    149884       3      0 recvfrom_nocancel(0x6, 0x105D85A20, 0x42)         = 66 0
 1806/0x1dd3:    149886       3      1 select_nocancel(0x7, 0x7FFF5FBFADB0, 0x0)         = 0 0
 1806/0x1dd3:    149888       2      0 kevent(0x5, 0x7FFF5FBFA860, 0x1)      = 0 0
 1806/0x1dd3:    149889       2      0 kevent(0x5, 0x0, 0x0)         = 1 0
 1806/0x1dd3:    149890       2      0 kevent(0x5, 0x0, 0x0)         = 0 0
 1806/0x1dd3:    149900  389953      9 kevent(0x5, 0x0, 0x0)         = 1 0
 1806/0x1dd3:    149904      60      1 kevent(0x5, 0x0, 0x0)         = 1 0
 1806/0x1dd3:    149909       7      3 recvfrom_nocancel(0x6, 0x7FFF5FBFAE30, 0x1C)      = 28 0
 1806/0x1dd3:    149913       7      2 recvfrom_nocancel(0x6, 0x105D8FD10, 0x2E)         = 46 0
 1806/0x1dd3:    149921       9      4 select_nocancel(0x7, 0x7FFF5FBFADB0, 0x0)         = 0 0
 1806/0x1dd3:    149928       5      1 kevent(0x5, 0x7FFF5FBFA860, 0x1)      = 0 0
 1806/0x1dd3:    149932       3      1 kevent(0x5, 0x0, 0x0)         = 1 0
 1806/0x1dd3:    149932       2      0 kevent(0x5, 0x0, 0x0)         = 0 0
 1806/0x1dd3:    149945  439745     12 kevent(0x5, 0x0, 0x0)         = 1 0
 1806/0x1dd3:    149952      13      3 kevent(0x5, 0x0, 0x0)         = 1 0
 1806/0x1dd3:    149961      15      6 recvfrom_nocancel(0x6, 0x7FFF5FBFAE30, 0x1C)      = 28 0
 1806/0x1dd3:    149969      11      4 recvfrom_nocancel(0x6, 0x105D8FD10, 0x2A)         = 42 0
 1806/0x1dd3:    149981      16      8 select_nocancel(0x7, 0x7FFF5FBFADB0, 0x0)         = 0 0
 1806/0x1dd3:    149987       9      3 kevent(0x5, 0x7FFF5FBFA860, 0x1)      = 0 0
 1806/0x1dd3:    150002      19      9 sendto_nocancel(0x6, 0x105D8FCF0, 0x1C)       = 28 0
 1806/0x1dd3:    150011      12      5 sendto_nocancel(0x6, 0x105D8FCF0, 0x1C)       = 28 0
 1806/0x1dd3:    150025      22     10 close_nocancel(0x5)       = 0 0
 1806/0x1dd3:    150095      26     18 socket(0x2, 0x1, 0x6)         = 5 0
 1806/0x1dd3:    150109      12      4 setsockopt(0x5, 0xFFFF, 0x1022)       = 0 0
 1806/0x1dd3:    150113       9      2 fcntl(0x5, 0x3, 0x0)      = 2 0
 1806/0x1dd3:    150117       7      2 fcntl(0x5, 0x4, 0x6)      = 0 0
 1806/0x1dd3:    150191      81     73 connect(0x5, 0x7FFF5FBFB9F0, 0x10)        = -1 Err#36
 1806/0x1dd3:    150217  689426     21 select(0x6, 0x7FFF5FBFB730, 0x7FFF5FBFB6B0, 0x7FFF5FBFB630, 0x7FFF5FBFB7D0)       = 1 0
 1806/0x1dd3:    150226      72      2 getsockopt(0x5, 0xFFFF, 0x1007)       = 0 0
 1806/0x1dd3:    150230       6      2 getpeername(0x5, 0x7FFF5FBFB780, 0x7FFF5FBFB6FC)      = 0 0
 1806/0x1dd3:    150232       4      1 getsockname(0x5, 0x7FFF5FBFB700, 0x7FFF5FBFB6FC)      = 0 0
 1806/0x1dd3:    150332      23     17 open_nocancel("/opt/local/share/curl/curl-ca-bundle.crt\0", 0x0, 0x1B6)       = 7 0
 1806/0x1dd3:    150339       6      2 fstat64(0x7, 0x7FFF5FBFB440, 0x7FFF5FBFB50C)      = 0 0
 1806/0x1dd3:    150350      14      9 read_nocancel(0x7, "##\n## lib/ca-bundle.crt -- Bundle of CA Root Certificates\n##\n## Certificate data from Mozilla as of: Thu Nov 22 00:12:21 2012\n##\n## This is a bundle of X.509 certificates of public Certificate Authorities\n## (CA). These were automatically extracted from Mo", 0x1000)      = 4096 0

So it looks like it isn't a direct file access that's failing in the process but socket send. I'm following some instructions to try set up /dev/urandom and other non-file paths into the chrooted environment, but again, is there any way to know what socket socketpair(0x1, 0x1, 0x0) is rather than just working in the dark?

Update 2

It's a known problem on OSX to do a DNS lookup inside a chroot: https://lists.macosforge.org/pipermail/macports-dev/2011-April/014565.html

Apparently I need to run a mDNSResponder service at /path/to/chroot/var/run/mDNSResponder

I'm having some issues with requests for pages not getting a response after requests take a long time to process.

I have nginx setup to use php-fpm. I have two pools setup in PHP-FPM. One pool for general web page requests, one pool to serve up image and other large files.

From my php-fpm config file:

[www]
listen = var/run/php54/php-fpm-www.sock
pm = dynamic
pm.max_children = 20
pm.start_servers = 4
pm.min_spare_servers = 4
pm.max_spare_servers = 20
pm.max_requests = 200


[www-images]

listen = var/run/php54/php-fpm-images.sock

pm = dynamic
pm.max_children = 5
pm.start_servers = 1
pm.min_spare_servers = 1
pm.max_spare_servers = 2
pm.max_requests = 40

Nginx is configured to use these two separate pools, with requests for images stored in Amazon S3 going through the 'www-images' pool to be resized to the requested size. From my nginx config file:

location ~* ^/proxy  {
    try_files $uri @404;
    fastcgi_pass   unix:/opt/local/var/run/php54/php-fpm-images.sock;
    include       /opt/local/etc/nginx/fastcgi.conf;
}

location  / {
    try_files $uri /routing.php?$args;
    fastcgi_pass   unix:/opt/local/var/run/php54/php-fpm-www.sock;
    include       /opt/local/etc/nginx/fastcgi.conf;
}   

Because I'm testing on a terrible internet connection these requests are timing out in PHP, which is expected.

2013/01/20 15:47:34 [error] 77#0: *531 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 127.0.0.1, server: example.com, request: "GET /proxy/hugeimage.png HTTP/1.1", upstream: "fastcgi://unix:/opt/local/var/run/php54/php-fpm-images.sock:", host: "example.com", referrer: "http://example.com/pictures"

What's not expected and I'd like to resolve is that any requests that should be going to the 'www' pool are timing out with nginx not getting a response from PHP-FPM.

2013/01/20 15:50:06 [error] 77#0: *532 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 127.0.0.1, server: example.com, request: "GET /pictures HTTP/1.1", upstream: "fastcgi://unix:/opt/local/var/run/php54/php-fpm-www.sock:", host: "example.com"

After a couple of minutes requests to the 'www' pool start working again, without any action on my part.

I thought that using separate pools should mean that even if one pool has issues with requests taking a long time, the other pool should remain unaffected.

So my question is; how do I isolate the two pools, so that one pool being overwhelmed by requests that are timing out, doesn't affect the other pool.

To clarify, it is by design that I want to limit the number of requests that can be made at once through the 'www-images' pool. Although in practice this limit will hardly ever be reached (due to caching of the files downloaded from S3 to the server), if there is an unusual situation where that pool reaches it's limit, I want the www pool to continue functioning, as that is where the sites functionality actually sits.