Ian's questions

I'm trying to pass headers from php code back to the apache accesslog by using HTTP headers, like so:

Header note X-Userid userid
Header unset X-Userid

LogFormat "%h %l %{userid}n %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" combined_with_php_userid
CustomLog /var/log/apache2/access_log combined_with_php_userid

With mod_php, the userid is inserted into the log as expected, and the header is unset before being sent to the client.

When running via php-fpm, using the following line, the userid is not inserted in the log and is not unset in the client HTTP headers.

ProxyPassMatch ^/(.*\.php(/.*)?)$ fcgi://127.0.0.1:9001/var/html/$1

Originally I was using apache_note but this is only available with mod_php. I found the above as a solution for passing data from PHP to Apache/php-fpm or nginx, but it doesn't seem to work with php-fpm.

Is there something I need to enable or set to get Header unset working under php-fpm?

Virtual Host Config:

<VirtualHost *:80>
    ProxyPassMatch ^/(.*\.php(/.*)?)$ fcgi://127.0.0.1:9001/web/ee2/sites/site.com/$1
    ServerAdmin [email protected]
    DocumentRoot /web/ee2/sites/site.com
    ServerName site.dev

    Header note X-Userid userid
    Header unset X-Userid

    ErrorLog  /var/log/apache2/site.dev-error_log
    LogFormat "%h %l %{userid}n %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" combined_with_php_userid
    # also tried: # LogFormat "%h %l %{X-Userid}i %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" combined_with_php_userid
    CustomLog /var/log/apache2/searchenginenews.com-access_log combined_with_php_userid

    <Directory /web/ee2/sites/site.com>
        AllowOverride All
        Require all granted
    </Directory>

</VirtualHost>

If I have two workers listening on a single tube and a job comes through, will both workers attempt to handle the same job, or will it get claimed only by one?

I have a simple shell script that:

1. Makes a directory on the remote machine.
2. Copies a .epub file to the remote machine's new directory.
3. Runs kindlegen to convert the .epub to a .mobi.
4. The remote server copies the new .mobi file back to the originating server.

Both servers are configured with SSH passwordless login via public key.

Running the script at the command line works well. Running it from the server from a PHP shell_exec command will execute the script, but the ssh and scp commands do not seem to get executed, or at least don't output anything, even if I use a -v flag.

Here's the PHP function that calls the shell script:

function mobi_watermark($userid, $email, $epubpath)
{

    $outpath = "/tmp/epub$userid/"; # the book gets marked for each user, store each epub under a unique work folder to avoid collision
    shell_exec("chmod -R 777 ".$outpath); # web server creates files under user "nobody".
    shell_exec("del.sh"); # deletes old files if they exist
    shell_exec("rep.sh $userid $email $epubpath"); # creates the initial .epub and tmp dir
    shell_exec("chmod -R 777 ".$outpath); 
    shell_exec("kindle.sh $outpath"); # this is the mobi conversion, where the problem is
    $this->mobi_ufa_download('ufa-187.mobi',$outpath);
}

And here's the shell script with the failing commands: [ kindle.sh ]

#!/usr/local/bin/bash
# uses the same /tmp$userid path as the other scripts, but on the remote server
TMPPATH=$1 
cd $TMPPATH
# create remote dir
ssh -v user@server 'mkdir -v '$TMPPATH
# copy the source epub file to remote
scp -v $TMPPATHfile-name.epub user@server:$TMPPATH
# perform the conversion and copy the .mobi result back to originating server
/usr/bin/ssh -v user@server 'cd '$TMPPATH'; /home/username/kindlegen/kindlegen ./file-name.epub; scp -v file-name.mobi user@originating-server:'$TMPPATH';rm -rf '$TMPPATH 

If I run this series of commands either from a script file or as individual commands from the command line myself, it all works perfectly.

When run via the php script, I can see script output if I decide to echo test commands, and scp will report an error if I break the source filename, but simply nothing happens for scp or ssh commands if they are otherwise correct. There's not even any verbose debug output from the -v flag.

I must be missing something obvious? I'm guessing it's because PHP is using user 'nobody' which SSH does not like. Is there a way around that, if that is the issue?

I'm running MAMP on OS X 10.7. I've set up a virtual host in Apache, like so:

<Directory "/sites/example.com/">
Allow From All
AllowOverride All
</Directory>
<VirtualHost *:80>
    ServerName "example.local"
    ServerAlias "www.example.local"
    DocumentRoot "/sites/example.com"
</VirtualHost>

In /etc/hosts I've added this:

127.0.0.1 example.local

Everything works fine, and up to maybe a week or two ago was very fast. I'm not sure what changed, but now the DNS Lookup portion of the request takes upwards of 5 seconds nearly every time the page is loaded, as reported by both Firebug and Chrome.

Pinging the local address looks good:

->ping example.local
PING example.local (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: icmp_seq=0 ttl=64 time=0.073 ms

Here's what Chrome shows:

If I refresh immediately after the first connection, the DNS lookup appears to be cached. But refreshing again a short time later causes the slow lookup again.

Any help would be appreciated!

I'm testing out Google Apps for Business for my company, and I'd like to send mail both to the Google Apps MX servers as well as my existing server. This way I can test the functionality of Google Apps without disrupting my existing users and systems.

Is it as simple as adding the Google Apps MX records as additional MX records with higher cost (lower priority) than my existing MX records?

I run a cron job that requests a snapshot from a remote webcam at a local address:

wget http://user:[email protected]/snapshot.cgi

This creates the files snapshot.cgi, snapshot.cgi.1, snapshot.cgi.2, each time it's run.

My desired result would be for the file to be named similar to file.1.jpg, file.2.jpg. Basically, sequentially or date/time named files with the correct file extension instead of .cgi.

Any ideas?

I'm attempting to get gltail set up on my local system to monitor logs remotely.

I've got all the dependencies installed, but when I go to run gltail ...

./gl_tail configfile ../config.yaml

I get this output:

Missing gem net-ssh.
Ubuntu:
  sudo gem install -y net-ssh -r

.. which I've done a number of times. Here is the output:

dev@ubuntu:~/Desktop/Fudge-gltail-e5b252d/bin$ sudo gem install net-ssh
Successfully installed net-ssh-2.0.15
1 gem installed
Installing ri documentation for net-ssh-2.0.15...
Installing RDoc documentation for net-ssh-2.0.15...

But when I go to run it again, I get the same missing gem net-ssh error.

dev@ubuntu:~/Desktop/Fudge-gltail-e5b252d/bin$ which ruby
/usr/bin/ruby
dev@ubuntu:~/Desktop/Fudge-gltail-e5b252d/bin$ which gem
/usr/bin/gem

Why isn't net-ssh being seen as properly installed?

An affiliate marketing person has recently started sending my website strange traffic.

1) The visitor hits my site at the affiliate's special location, which performs a 302 redirect to the homepage of the site.

2) The visitor follows the redirect to the homepage of the site, where they are served another 302 redirect and then disappear.

Here is what the logs look like specifically:

www.example.com 74.197.XX.XXX - - [20/Aug/2009:16:15:28 -0400] "GET /go HTTP/1.1" 302 298 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; Zune 2.0)"
www.example.com 74.197.XX.XXX - - [20/Aug/2009:16:15:28 -0400] "GET / HTTP/1.1" 302 460 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; Zune 2.0)"

When I personally visit the first URL, I get redirected and then when I get the homepage, I get the regular HTML content and then download all the usual page assets ( js / css / images, etc ).

But these visitors, and there are hundreds of them all with varying ip addresses and user agents, etc, all follow this strange pattern of two 302 redirects and then poof.

Following the referer of some of the visits, the source of the traffic looks pretty bad. Your typical pop-under BS.

My primary concern is, what is happening to these 'visitors' with that second 302 redirect? Why is my server doing a 302?

The only thing in the code that would result in a 302 redirect from the homepage would be if the visitor has cookies disabled. But in that case, I would see a whole series of requests before the browser timed out due to the 'this site is redirecting in a way that will never complete' issue. (Basically, if I don't see they have a site cookie, they get redirected to 'refresh' the session which then lets me see the cookie.)

Regardless, it's unlikely that all 300+ visitors who came in like this would have cookies disabled.

Are there any outside influences that could cause my server to do a redirect? Is there anyway to configure apache logging to include the location that the 302 points to?

I don't have root access on my hosted web server, but I would like to use tools like MultiTail to monitor log files that are generated on my web host.

Is there a way I can pipe the log output on my remote host ( I have ssh access ), to my locally running MultiTail?

The local machine is running Ubuntu 9.04. The remote machine is running FreeBSD 6.

==

The answer in a nut-shell:

1) Set up key-based login for ssh.

2) Execute the following command:

multitail -l "ssh -l USER HOST tail -f ./www_logs/access_log" -l "ssh -l USER2 HOST2 tail -f ./www_logs/access_log"

I sometimes monitor apache and php error logs using tail under FreeBSD. Is there any way to get colorized output, either using tail or some other command line app?

Alternatively, what is your favorite way to monitor the various web-related logs in realtime?

Is there an upper limit to the total number of files stored in a single subdirectory in FreeBSD 6?

The filesystem is ufs

I have Ubuntu 9.04 installed in VMWare 6.52. When I login to the machine, the login screen has a strange resolution problem.

Here's an example screen shot of the issue:

(ugh, cant get image shack link to work here.. please follow the link to see the screen shot:

Screen shot

The white line that I drew in there indicates the invisible mouse border. In other words, I can't move my mouse all the way across the entire black area. It stops at where I drew in the white border. The white border does not actually exist on the login screen.

After I log in, the resolution of the desktop appears fine.

I have Ubuntu server running in a VMWare 6.5.2 virtual machine. I made a straight file copy of the entire VM folder structure to my laptop from my desktop and then booted it up.

After the VM guest OS boots, things are a little wonky. In particular, mysql won't start, and all my localhost'd files come up 404, even the root index.

Any ideas?

Edit - To clarify, I copied the Virtual Machine files located in the Host Windows computer's file system from my Desktop computer to my Laptop computer, and then opened the copied files in VMware on the laptop.

Here are the mysql error logs from syslog:

Jul 23 11:51:03 ubuntu mysqld_safe[9194]: started
Jul 23 11:51:03 ubuntu mysqld[9197]: 090723 11:51:03  InnoDB: Started; log sequence number 0 8539038
Jul 23 11:51:03 ubuntu mysqld[9197]: 090723 11:51:03 [ERROR] Can't start server: Bind on TCP/IP port: Cannot assign requested address
Jul 23 11:51:03 ubuntu mysqld[9197]: 090723 11:51:03 [ERROR] Do you already have another mysqld server running on port: 3306 ?
Jul 23 11:51:03 ubuntu mysqld[9197]: 090723 11:51:03 [ERROR] Aborting
Jul 23 11:51:03 ubuntu mysqld[9197]: 
Jul 23 11:51:03 ubuntu mysqld[9197]: 090723 11:51:03  InnoDB: Starting shutdown...
Jul 23 11:51:04 ubuntu mysqld[9197]: 090723 11:51:04  InnoDB: Shutdown completed; log sequence number 0 8539038
Jul 23 11:51:04 ubuntu mysqld[9197]: 090723 11:51:04 [Note] /usr/sbin/mysqld: Shutdown complete
Jul 23 11:51:04 ubuntu mysqld[9197]: 
Jul 23 11:51:04 ubuntu mysqld_safe[9218]: ended
Jul 23 11:51:17 ubuntu /etc/init.d/mysql[9360]: 0 processes alive and '/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf ping' resulted in
Jul 23 11:51:17 ubuntu /etc/init.d/mysql[9360]: ^G/usr/bin/mysqladmin: connect to server at 'localhost' failed
Jul 23 11:51:17 ubuntu /etc/init.d/mysql[9360]: error: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)'
Jul 23 11:51:17 ubuntu /etc/init.d/mysql[9360]: Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists!
Jul 23 11:51:17 ubuntu /etc/init.d/mysql[9360]: 

Here are the apache error logs. Regardless of what localhost url I try to open, the same error line is output:

[Thu Jul 23 11:57:38 2009] [error] [client 127.0.0.1] File does not exist: /htdocs
[Thu Jul 23 11:59:53 2009] [error] [client 127.0.0.1] File does not exist: /htdocs
[Thu Jul 23 11:59:59 2009] [error] [client 127.0.0.1] File does not exist: /htdocs
[Thu Jul 23 12:00:04 2009] [error] [client 127.0.0.1] File does not exist: /htdocs

I've got an Ubuntu LAMP install running in a VMWare virtual machine. I'm able to connect to the mysql server from inside the VM from the command line, but I'm not able to connect to it from inside the Host OS via the VM's ip address. The error message returned by the mysql server through SQLYog is ...

Error No. 1130

Host 'monolith.home' is not allowed to connect to this MySQL server

... where 'monolith.home' is the computer name and domain of the Host computer.

So the mysql port appears open, it is just rejecting my connection. Where do I change the kind of access restriction?

After rebooting my fairly freshly installed Ubuntu Desktop 9.04 vmware image today, the desktop comes up with the screen offset (approximately) 50 pixels to the left. So the left-most 50 pixels on the desktop are 'outside' the VMWare host window (not visible), and the right-most 50 pixels inside the VMWare host window are black. So the entire screen is shifted left by 50 pixels, revealing a "black background".

The login screen also doesn't show the complete login box. The login box is cut off mid-way through the last "e" in "Username".

The offset remains after changing the screen resolution.

Hmm.. Just rebooted again. The Login box is still cut off, but the screen is no longer offset.

What's the deal with this?

Host machine is Windows 7 7068

When I edit the crontab using crontab -e it opens the crontab in vi. I'd prefer it to use nano. How can I change this?

EDIT:

For some reason the export command returns command not found. And changing the EDITOR env value didn't work for crontab -e.

The following command worked on my system:

setenv VISUAL /usr/local/bin/nano

I'm working on getting DKIM installed on my outgoing mail server, to help my email deliverability to my Yahoo clients (all legit emails, etc, no spamming).

I've got DKIM-Signature: and DomainKey-Signature: headers being generated, but a test mailer to my Yahoo account had this line in the headers:

Authentication-Results: mta250.mail.re2.yahoo.com from=example.com; domainkeys=permerror (no key); from=example.com; dkim=permerror (no key)

Any ideas what I may have missed?

Successfully delivering email to my clients is a pretty high priority at my company. We are of course fully opt-in, paid clients, no possibility of spam, the whole thing. The information we email is specifically requested from us by the recipient.

We have done some of the easy stuff like setting up SPF records on our outgoing mail servers.

What other steps can we take to ensure that our email goes through, in terms of email server setup and configuration ( similar to SPF, etc )

Alternatively, are there 3rd party email services that we can pay for that already manage these issues?

I'm following the instructions at http://www.ntop.org/download.html.

At some point during the ./configure phase of the install, I get this error message:

configure: error: Unable to find RRD at /usr/local/rrdtool: please use --with-rrd-home=DIR

I run which rrdtool and I get:

/usr/local/bin/rrdtool

...so I run: ./configure --with-rrd-home=/usr/local/bin/rrdtool

The installation continues, until it fails at the same spot, same error. I've tried a bunch of varying paths, like:

`./configure --with-rrd-home=/usr/local/bin/`
`./configure --with-rrd-home=/usr/local/bin`
`[...]`
`./configure --with-rrd-home=/usr`

... as I read in a suggestion while Googling the problem.

If it matters, I don't have root access on this machine. It's a dedicated server at my web host.

Any suggestions?

I'm trying to locate all copies of example.filename on my FreeBSD server. What's the best / easiest / most efficient way to do this?