Max's questions

I'm using Nginx 1.2.4 in combination with PHP-FPM 5.4.8 with a fastcgi pass and trying to pass custom parameters to PHP. Here are the options I have found so far:

• using the env directive to set an environment variable in nginx and fetch it with getenv() or $_ENV from PHP. The problem is that env only operates in main context whereas I need to set the parameter in server context.

• using the fastcgi_param directive as it's designed for that.

I have tried changing the values of some parameters using fastcgi_param but was unsuccessful:

nginx:

fastcgi_param PATH_INFO "/var/tmp";

PHP:

I checked all the predefined arrays I could find:

echo '<pre>';
echo "\n".'$GLOBALS'."\n"; var_dump($GLOBALS);
echo "\n".'$_SERVER'."\n"; var_dump($_SERVER);
echo "\n".'$_GET'."\n"; var_dump($_GET);
echo "\n".'$_POST'."\n"; var_dump($_POST);
echo "\n".'$_FILES'."\n"; var_dump($_FILES);
echo "\n".'$_REQUEST'."\n"; var_dump($_REQUEST);
echo "\n".'$_SESSION'."\n"; var_dump($_SESSION);
echo "\n".'$_ENV'."\n"; var_dump($_ENV);
echo "\n".'$_COOKIE'."\n"; var_dump($_COOKIE);
echo "\n".'$php_errormsg'."\n"; var_dump($php_errormsg);
echo "\n".'$HTTP_RAW_POST_DATA'."\n"; var_dump($HTTP_RAW_POST_DATA);
echo "\n".'$http_response_header'."\n"; var_dump($http_response_header);
echo "\n".'$argc'."\n"; var_dump($argc);
echo "\n".'$argv'."\n"; var_dump($argv);
echo '</pre>';

The only one to show PATH_INFO was $_SERVER:

var_dump($_SERVER); // ["PATH_INFO"]=> string(0) ""

But the value I set in nginx isn't taken into account.

Q1: Do I need to configure anything at nginx/php level for the fastcgi_param directive to take effect?

Q2: Is the fastcgi_param directive limited to a predefined list of parameters (e.g. I can set PATH_INFO but not custom variables of my own such as FOO)?

Q3: If yes to Q2: is there a way to pass custom parameters to PHP from Nginx?

I need to include dynamic content into static pages at web server level. The 2 options I have found so far are Server Side Include (SSI) and Edge Side Include (ESI).

While the origins of SSI seem old and obscure (that cached page from 95 from University of Illinois seems to be the reference, apparently it's from the NCSA httpd webserver which used to power about 95% of the net), that of ESI seem more recent and rejoicing (w3 specifications from 2001, written mostly by guys from Akamai).

Also, I keep hearing about Varnish+ESI and I'm wondering whether that should therefore be the way to go. However I already have a setup in place with nginx, which only supports SSI, and would like to follow the KISS principle and avoid using Varnish if at all possible.

For my immediate use case, which will be to include a dynamic user bar at the top of each pages, I believe SSI will do the job. However I'm concerned that as my website grows, I will need features only supported by ESI which will force me to re-design everything, which brings me to my question (finally the reader says):

What are the main features not supported by SSI that would make you choose ESI (and vice versa) ?

Using nginx 1.2.1 I am able to use add_header under some circumstances.

Here is a working example:

server {
    listen       80;
    server_name  localhost;
    root /var/www;
    add_header Name1 Value1;      <=== HERE
    add_header Name2 Value2;      <=== HERE

    location / {
        echo "Nginx localhost site";
    }
}

The result

GET /
HTTP/1.1 200 OK
Name1: Value1
Name2: Value2

Here is something that doesn't work: if I use the add_header directive inside location, the other add_header directives under server are ignored:

server {
    listen       80;
    server_name  localhost;
    root /var/www;
    add_header Name1 Value1;      <=== HERE
    add_header Name2 Value2;      <=== HERE

    location / {
        add_header Name3 Value3;  <=== HERE
        add_header Name4 Value4;  <=== HERE
        echo "Nginx localhost site";
    }
}

The result

GET /
HTTP/1.1 200 OK
Name3: Value3
Name4: Value4

The documentation says that both server and location are valid context and doesn't state that using add_header in one prevents using it in the other.

Q1: Do you know if this is a bug or the intended behaviour and why?

Q2: Do you see other options to get this fixed than using the HttpHeadersMoreModule module?

As far as I can see Nginx supports by default 2 log files: error_log (tracks issues related to the Nginx server itself) and access_log (tracks requests processed by Nginx). Whilst it is possible to control the format of access_log with the log_format directive, I have been unsuccessful at finding a way of logging only certain requests to a separate file, and therefore would like to ask the question on SF as a reference for future readers:

Is there a way to log certain requests to a different log file than the one defined by access_log?

FYI the reason behind this question is that I have a rule that denies access to unwanted crawlers with a 200 (because 403 would give them a hint that they're being blocked), and filtering those requests out of the access_log becomes more difficult.

I've created a script to cleanup InnoDB storage engine. Everything works fine except that creating the ib_logfile0 and ib_logfile1 files when innodb_log_file_size is set to 1GB takes a while which causes /etc/init.d/mysql to timeout and report a failure, although in the background MySQL ends up starting normally.

Starting mysql

$ /etc/init.d/mysql start
* Starting MariaDB database server mysqld  [fail] 

Meanwhile in the logs

$ tail -f /var/log/mysql.err
120426 11:19:55  InnoDB: Log file ./ib_logfile0 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile0 size to 1024 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Progress in MB: 100 200 300 400 500 600 700 800 900 1000
120426 11:20:07  InnoDB: Log file ./ib_logfile1 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile1 size to 1024 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Progress in MB: 100 200 300 400 500 600 700 800 900 1000
[...]
120426 11:20:24 [Note] /usr/sbin/mysqld: ready for connections.
# Version: '5.5.23-MariaDB-mariadb1~oneiric-log'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution

I am looking for a way to prevent the startup script from timing out. When I look at /etc/init.d/mysql, I can see the ${MYSQLD_STARTUP_TIMEOUT} variable but don't know where it is set. I've also been looking in the documentation for such an option but couldn't find it.

Q1: Can I specify a custom startup timeout as a parameter to /etc/init.d/mysql ?
Q2: What option can I modify to change the server startup timeout in the configuration?

We're experiencing performance issues on our website, and after reviewing our munin graphs, the only metrics we've found in sync are Available entropy and MySQL slow queries, with the latter influenced by our number of logged in users:

Based on the wikipedia entropy page, my understanding is that entropy is the amount of randomness (here measured in bytes) that the system can use for various tasks, mainly cryptography and functions that require random input.

Since the peaks in available entropy and MySQL slow queries are occurring in sync and at regular interval, that the number of MySQL slow queries is proportional to our number of Drupal users whereas the peaks in available entropy seem to be much more constant and less proportional to these 2 metrics, we're thinking available entropy is the reflect of a root cause which, combined with the traffic to our website, is causing those slow queries (and not the opposite, slow queries influencing the entropy). Accordingly:

Q: What underlying problem do you think could cause regular peaks in available entropy that could have an influence on MySQL's ability to process queries?

I can see the list of core and other modules loaded by apache using the below command:

$ apache2ctl -t -D DUMP_MODULES
Loaded Modules:
 core_module (static)
 mpm_prefork_module (static)
 http_module (static)
 so_module (static)
 auth_basic_module (shared)
 auth_digest_module (shared)

In order to optimize the performance of the server (save memory on each Apache thread essentially), I would like to disable unused modules. The Apache documentation provides information about this modules at the following page.

Most of the time, the information provided for each module is enough to determine whether I should disable a module or not. For instance I know I don't use LDAP authentication so I can safely disable authnz_ldap_module.

However for some modules it is a bit less obvious. For instance the mime_magic module is used as a "second line of defense" for cases that mod_mime can't resolve, which can't be determined theoretically as it depends on what files will get served by Apache.

Q: Is there a way to log to a file the list of modules that Apache effectively used to process a request?

If I could log such information, I would just let the web server run for some time, then get the list of modules that were used to process all the requests received in the meantime, and disable those that don't appear in that list.

We're running a heavy Drupal website that performs financial modeling. We seem to be running into some sort of memory leak given the fact that overtime the memory used by apache grows while the number of apache processes remains stable:

We know the memory problem is coming from apache/PHP because whenever we issue a /etc/init.d/httpd reload the memory usage drops (see above screenshot and below CLI outputs):

Before httpd reload

$ free
             total       used       free     shared    buffers     cached
Mem:      49447692   45926468    3521224          0     191100   22609728
-/+ buffers/cache:   23125640   26322052
Swap:      2097144     536552    1560592

After httpd reload

$ free
             total       used       free     shared    buffers     cached
Mem:      49447692   28905752   20541940          0     191360   22598428
-/+ buffers/cache:    6115964   43331728
Swap:      2097144     536552    1560592

Each apache thread is assigned a PHP memory_limit of 512MB which explains the high memory usage depiste the low volume of requests, and a max_execution_time of 120 sec which should terminate threads which execution is taking longer, and should therefore prevent the constant growth in memory usage we're seeing.

Q: How could we investigate what is causing this memory leak?

Ideally I'm looking for troubleshooting steps I can perform on the system without having to bother the dev team.

Additional info:

OS: RHEL 5.6
PHP: 5.3
Drupal: 6.x
MySQL: 5.6

FYI we're aware of the swapping issue which we're investigating separately and has nothing to do with the memory leak which we've observed before the swapping started to occur.

I'm trying to import passwords from an LDAP to a MySQL database. When I look at the userPassword attribute for the known password test I get this:

userPassword:: e01ENX1DWTlyelVZaDAzUEszazZESmllMDlnPT0=

The password at this stage is base64 encoded, after decoding I get this:

{MD5}CY9rzUYh03PK3k6DJie09g==

-It doesn't look like the CY9.... string is an MD5 hash because it isn't 32-character long (actually 128-bit).
-It doesn't seem to be base64 encoded either as I don't get test after decoding.
-Also when I generate my own MD5 hash out of test I get 098f6bcd4621d373cade4e832627b4f6.

I found this perl script (second one on the page) which does generate {MD5}CY9rzUYh03PK3k6DJie09g== from test but when I look at the script I don't see what I'm missing as it does look like the md5 hash gets base64 encoded before being concatenated with {MD5}:

$ctx = Digest::MD5->new;
$ctx->add('secret');
$hashedPasswd = '{MD5}' . encode_base64($ctx->digest,'');

Can someone explain the step to get from test to CY9rzUYh03PK3k6DJie09g==?

I would like to analyze mysql traffic. Right now, all mysql requests are sent to the MySQL unix socket:

unix  2      [ ACC ]     STREAM     LISTENING     3734388  15304/mysqld        /var/run/mysqld/mysqld.sock

I'm trying to disable that socket to force MySQL to use the network socket instead on the loopback. I tried commenting out all the socket directives in the my.cnf and debian.cnf files and restarted MySQL but it made no difference.

How can I disable the MySQL unix socket to force MySQL over the network?

additional info: I'm running MySQL 5.1 on ubuntu 10.04.

Precisions on the question
Since plenty of people suggested enabling the network socket I would like to clarify my question by pointing out that the bind address was already enabled with bind-address = 127.0.0.1 and that a listening connection is available:

tcp        0      0 127.0.0.1:3306          0.0.0.0:*               LISTEN      15601/mysqld 

Still I see no connections attempt to 127.0.0.1:3306 coming from my webapp (Drupal website).

Updated with the answer

It appears indeed that the issue is coming from the mysqli connector that Drupal uses (in .ht_config.php for those who are interested). It was set: mysqli://drupal:***@localhost/drupal, changing localhost to 127.0.0.1 fixed the issue (i.e. Drupal is now making connections to the network socket).

I want to find out what was added to a log file between 2 tasks. Right now the way I do it is:

content of test.log initially

aaa
bbb

saving content of the log

$ cp test.log test.log.before

doing something

content of test.log after doing something

aaa
bbb
ccc

saving content of the log

$ cp test.log test.log.after

showing the difference

$ diff test.log.before test.log.after
2a3
> ccc

What I don't like with this method is that I get some noise from diff (2a3 and >) whereas I just want ccc.

Q: Is there a way I can just get the difference between the 2 log files (i.e. what is new in test.log.after, ccc)?

I had a look at diff's man page but I don't see anything obvious there.

I've enabled the apache_ plugins on a munin node:
ln -sv /usr/share/munin/plugins/apache_* /etc/munin/plugins/

After restarting the node with service munin-node restart here are the errors I'm getting:

$ munin-node-configure --suggest 2>/dev/null | grep "apache\|Plugin\|------"
Plugin                     | Used | Suggestions                            
------                     | ---- | -----------                            
apache_accesses            | yes  | no [apache server-status not found. check if mod_status is enabled]
apache_processes           | yes  | no [apache server-status not found. check if mod_status is enabled]
apache_volume              | yes  | no [apache server-status not found. check if mod_status is enabled]

However mod_status is already enabled:

$ a2enmod status
Module status already enabled

And restarting apache doesn't make a difference.

If I try to run the plugins manually here is what I get (I read that getting a U is bad news so at least that is consistent).

$ munin-run apache_accesses --debug
# Processing plugin configuration from /etc/munin/plugin-conf.d/munin-node
# Set /rgid/ruid/egid/euid/ to /110/65534/110 110 /65534/
# Setting up environment
# About to run '/etc/munin/plugins/apache_accesses'
accesses80.value U

$ munin-run apache_processes --debug
# Processing plugin configuration from /etc/munin/plugin-conf.d/munin-node
# Set /rgid/ruid/egid/euid/ to /110/65534/110 110 /65534/
# Setting up environment
# About to run '/etc/munin/plugins/apache_processes'
busy80.value U
idle80.value U
free80.value U

$ munin-run apache_volume --debug
# Processing plugin configuration from /etc/munin/plugin-conf.d/munin-node
# Set /rgid/ruid/egid/euid/ to /110/65534/110 110 /65534/
# Setting up environment
# About to run '/etc/munin/plugins/apache_volume'
volume80.value U

Does anybody know why I'm still getting the server-status not found message and how I can get rid of it?

Updated answer 1

Shane's suggestion was correct about setting a request handler using Location and SetHandler in the apache site. For more information on mod_status please refer to this page

I could verify that munin was effectively making the appropriate requests by looking at /var/log/apache2/access.log where I was getting this:

127.0.0.1 - - [10/Nov/2011:07:24:15 +0000] "GET /server-status?auto HTTP/1.1" 404 7774 "-" "libwww-perl/5.834

In my case setting the Location wasn't enough as I am running a Drupal site and the .htaccess combined with mod_rewrite were rewriting the requests. To fix it, I had to add the following line to my .htaccess

  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d
  RewriteCond %{REQUEST_URI} !=/favicon.ico
  RewriteCond %{REQUEST_URI} !=/server-status  # <= added this line
  RewriteRule ^(.*)$ index.php?q=$1 [L,QSA]

Please note that this doesn't represent a security problem since access to /server-status is restricted to 127.0.0.1 in the apache site.

Updated answer 2

It appears that adding the Location to the apache site wasn't needed after all since this is already defined in /etc/apache2/mods-enabled/status.conf. Btw, should you want to add the ExtendedStatus On directive, that's in that file that you should do it.

I'm implementing a deployment/rollback procedure using Jenkins 1.430. I would like to configure the deployment and the rollback steps as 2 separate jobs so as to be triggered independently if needed. However I would also like the rollback to be triggered automatically when the deployment fails.

When I go to the Post-build Actions configuration section of the deployment job, I can choose to Build other projects. If I check that setting I get 3 options:
-Trigger only if build succeeds
-Trigger even if the build is unstable
-Trigger even if the build fails

What I'm missing is the Trigger only if build fails option.

Do you know how this can be achieved?

The ldapmodify man page states that:

The default for ldapmodify is to modify existing entries

Yet when I try to import an LDIF file with ldapmodify I get the below error:

ldapmodify: modify operation type is missing at line X

Q1: Why, which arguments should I add to my ldapmodify command?

If I import an LDIF file using ldapadd and the entry already exists I get the below error:

ldap_add: Already exists (68)

This can be ignored using the -c switch (for continue), however ldap_add won't update existing entries. Instead, in order to update existing entries one should use ldapmodify, however ldapmodify won't add missing entries.

Q2: Is there a way to import an LDIF files by creating missing entries AND updating existing ones at the same time?

I'm having issues importing users with ldapadd and ldif files. The error I'm getting is:

ldap_add: Constraint violation (19)
additional info: structuralObjectClass: no user modification allowed

The users imported are all part of ou=People,dc=example,dc=org. The LDAP server already contains this base DN.

The /etc/ldap/slapd.d/cn=config/olcDatabase={1}hdb.ldif file contains the following ACL entry:

olcAccess: {2}to dn.base="ou=People,dc=example,dc=org" attrs=children by gr
 oup.exact="cn=Manager,ou=Roles,dc=example,dc=org" manage

The ldif file is imported as follows:

ldapadd -f import.ldif -xv -D "cn=drupal,ou=Apps,dc=example,dc=org" -h localhost -W

The cn=drupal,ou=Apps[...] entry is a member of cn=Manager,ou=Roles,dc=example,dc=org so accordingly it should have sufficient permissions to write (since manage is the highest level of permissions available).

When I issue the ldapadd command the import fails on the very first ldif entry. The full command output is then:

add objectClass:
    top
    person
    inetOrgPerson
add uid:
    John.Merrell
add mail:
    [email protected]
add cn:
    John D Merrell
add structuralObjectClass:
    inetOrgPerson
add entryUUID:
    65236c42-09b7-1020-9318-9fca7c043dfc
add creatorsName:
    cn=drupal,ou=Apps,dc=bidnetwork,dc=org
add createTimestamp:
    20110503095643Z
add userPassword:
    2678u8yyy
add givenName:
    John D
add sn:
    Merrell
add entryCSN:
    20110629121956.880164Z#000000#000#000000
add modifiersName:
    cn=drupal,ou=Apps,dc=bidnetwork,dc=org
add modifyTimestamp:
    20110629121956Z
adding new entry "[email protected],ou=People,dc=example,dc=org"
ldap_add: Constraint violation (19)
    additional info: structuralObjectClass: no user modification allowed

I've tested importing users that did or did not exist on the LDAP and I get the aforementioned error in either case.

Can someone explain the origin of the problem and how it may be circumvented?

I would like to convert an ext4 partition to ext3 partition without formatting the HDD.

Is this possible and recommended?

When I connect to my Ubuntu 10.04.2 LTS server I get the following banner:

25 packages can be updated.
15 updates are security updates.

However if I run package upgrades/updates nothing comes up:

$ sudo apt-get update
$ sudo apt-get upgrade
...
Reading package lists... Done
Building dependency tree       
Reading state information... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

After doing some research I found that issuing apt-get dist-upgrade could help but it makes no difference.

Any help is welcome.

I was expecting slappasswd to produce a fixed hash but it appears that the output is randomized as I never get the same output for the same input password:

$ slappasswd -s secret
{SSHA}mCXsPZkfgQYZr2mKHpy5Iav+2S2XlVU3
$ slappasswd -s secret
{SSHA}62oXsalJBuopYfHhi6hGp6AESuWNIVnd
$ slappasswd -s secret
{SSHA}0Ulc8+ugMi3NbTtWnclOFW1LKCqRdsT8

During authentication, how does slapd knows how to randomize the hash for the provided password in the same way so that it can match the password defined in the first place?

Possible Duplicate:
Ubuntu : borked my sudoers file, how can I fix it?

Problem description:

I have broken my /etc/sudoers file by making a typo in it. Now I can no longer use any sudo commands as the below error is thrown. Therefore I can't fix my sudoers file and am effectively stuck.

$ sudo nano /etc/sudoers
>>> /etc/sudoers: syntax error near line 39 <<<
sudo: parse error in /etc/sudoers near line 39
sudo: no valid sudoers sources found, quitting

I found few solutions saying I should boot into recovery mode (like the one below) which I would obviously like to avoid.
Ubuntu : borked my sudoers file, how can I fix it?

Questions:

-Are there any other ways to fix it without restarting ubuntu?
-If it is so easy to lock yourself out of sudo when modifying the sudoers file (everybody can do a typo), why isn't the sudoers config read from memory instead of the file directly? There would be a command (e.g. sudo-load) that would make sure the syntax is OK before loading the config into memory thus avoid this sort of problems.

Solutions:
There seems to be 2 ways of recovering it:
-if you have the root password you can simply issue a 'su' and then edit the file
-if you don't have the root password you need to restart into single user mode

I would like to get an overview of available solutions for having a remote GUI access to ubuntu server:
-do you recommend installing a basic GUI then administrating over VNC?
-is X11 forwarding a good option?
...

I know there are similar questions already but they all focus on 1 aspect of the problem whereas I think it would be good to get an overview.

Thanks