Home / server / Questions / 312952
Accepted
quanta
Asked: 2011-09-20 03:24:28 +0800 CST

Weird Extended Information on Nagios Web UI

  • 772

Below are screenshots when I click on any Service/Host Extended Information:

The weird info at the top:

At the top is weird info

The content of line begin with __gmon_start_libc.so...:

__gmon_start__libc.so.6fflushstrcpy__rawmemchrgmtime_rexitreaddirfopenstrrchr__strdupclosedirstrncpyputsputcharreallocstdinstrpbrklocaltime_rgmtimestrtokmmapfgetscallocstrlenmemsetstrstr__errno_locationstdoutmemcpyfcloserand__strtol_internalmallocstrcatstrcasecmpasprintfopendirgetenvsscanfsystemmunmapstrncasecmp__fxstatstrncatfreadlocaltimestrchrmktimeqsort__strtod_internal__ctype_toupper_loc__ctype_tolower_loc__xstatstrcmpstrerror__libc_start_mainsnprintf__strtoul_internalfreeGLIBC_2.3GLIBC_2.2.5

and the menus with poor formatting at the bottom:

And bottom with poor formatting

Click on any service command, it responses with a "400 Bad request" error.

The configuration in Apache:

<VirtualHost *:80>
    ScriptAlias /nagios/cgi-bin "/usr/local/nagios/sbin"
    ServerName      xx
    ErrorLog    logs/nagios.error_log
    <Directory "/usr/local/nagios/sbin">
       Options ExecCGI
       AllowOverride None
       Order allow,deny
       Allow from all
    </Directory>

    Alias /nagios "/usr/local/nagios/share"

    <Directory "/usr/local/nagios/share">
       Options None
       AllowOverride None
       Order allow,deny
       Allow from all
    </Directory>
</VirtualHost>

There is nothing in Apache's error log related to this.

I've updated to the latest version 3.3.1:

nagios -v

Nagios Core 3.3.1
Copyright (c) 2009-2011 Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 07-25-2011
License: GPL

but it remains.

The permisison of extinfo.cgi:

-rwxrwxr-x 1 nagios nagios 285528 Sep 15 17:52 extinfo.cgi

What should I take at look at first to debug this case?

UPDATE

I've added the ScriptLog directive to nagios virtual host:

ScriptLog logs/cgi_log

This file is created and chmod manually:

-rw-r--r-- 1 apache apache 0 Sep 20 11:46 /var/log/httpd/cgi_log

Go to browser to replicate this error but log is empty. Does anyone have another idea?

UPDATE 2

Here is the output when running extinfo.cgi from command line as apache user:

# REQUEST_METHOD=GET QUERY_STRING="?type=2&host=149&service=Disk_Free" REMOTE_USER="quanta" ./extinfo.cgi

UPDATE 3

You mentioned other CGIs that are working, are they also called .cgi?

Of course. All the remaining .cgi scripts in /usr/local/nagios/sbin/ work fine. E.g status.cgi:

Service Status Details For All Hosts

The way you've configured the ScriptLog should be printing info.

But it didn't. This log file is blank.

-rw-r--r-- 1 apache apache 0 Sep 20 11:46 /var/log/httpd/cgi_log

It looks like you have AllowOverride turned off for this directory, but perhaps the apache configuration allows it for other directories higher up? In that case make sure you haven't disabled ExecCGI or have any .htaccess files in any directory above the nginx sbin.

There is no .htaccess file in /usr/local/nagios. I also compared this configuration with another file which is working, there is no significant difference:

ScriptAlias /nagios/cgi-bin "/usr/lib64/nagios/cgi"

<Directory "/usr/lib64/nagios/cgi">
   Options ExecCGI
   AllowOverride None
   Order allow,deny
   Allow from all
   AuthName "Nagios Access"
   AuthType Basic
   AuthUserFile /etc/nagios/htpasswd.users
   Require valid-user
</Directory>

Alias /nagios "/usr/share/nagios"

<Directory "/usr/share/nagios">
   Options None
   AllowOverride None
   Order allow,deny
   Allow from all
   AuthName "Nagios Access"
   AuthType Basic
   AuthUserFile /etc/nagios/htpasswd.users
   Require valid-user
</Directory>

If that doesn't work I'd increase the apache log level:

I did it but there is nothing related to cgi in error log:

proxy_util.c(1873): proxy: worker proxy:reverse already initialized
proxy_util.c(1967): proxy: initialized single connection worker 0 in child 3586 for (*)
proxy_util.c(1854): proxy: grabbed scoreboard slot 0 in child 3587 for worker proxy:revers

proxy_util.c(1873): proxy: worker proxy:reverse already initialized
proxy_util.c(1967): proxy: initialized single connection worker 0 in child 3587 for (*)

If none of that points out the problem I'd probably strace apache after setting it to only run a single child and seeing the output of a working CGI vs a non-working CGI to see what the issue is.

Good point. I enclosed here the strace files.

Trace only open files: 

strace -e trace=open -o /tmp/*.strace /usr/sbin/httpd -X

Trace full:

strace -o /tmp/*.strace /usr/sbin/httpd -X

Take a look at if you can help.

apache-2.2 nagios

3 Answers

  1. Best Answer

    If you rename extinfo.cgi, will it run under another name? If the script will be executed after renaming, most likely some parts of your Apache config handle the extinfo.cgi name in some unintended way. E.g., in /etc/httpd/conf.d/satisfy.conf you have:

    <LocationMatch "/info(1|2).php">

    This section will actually be applied to all requests which contain a matching part anywhere in the URL (because the regexp pattern is not anchored). E.g., it would apply to /info1.php, /some/dir/info2.php, /dir/info1_php_anything/and/more (note that a dot in regex matches any character, you need to write \. to match a literal dot). Also, Location and LocationMatch sections are always applied at the last step and override everything else (.htaccess, Directory, Files), and sections outside a VirtualHost definition are applied to all virtual hosts.

    If your config files contain other similar sloppy LocationMatch patterns, maybe some of them match /nagios/cgi-bin/extinfo.cgi and break CGI handling just for this particular script name.

    • 4

  2. The ScriptAlias directive is provided by mod_alias, but files in directories marked with this directive are processed by mod_cgi.

    Check if your mod_cgi is properly loaded.

    • 0

  3. I'm pretty confident this is a configuration issue, probably related to AddHandler. The output you provided as @Vladimir Blaskov also pointed out indicates apache doesn't think this is a cgi. You mentioned other CGIs that are working, are they also called .cgi? Or are they just in the directory that allows ExecCGI with no suffix? Also from the docs:

    http://httpd.apache.org/docs/current/mod/mod_cgi.html

    http://httpd.apache.org/docs/2.0/howto/cgi.html#configuring

    The way you've configured the ScriptLog should be printing info. If I were in your situation I'd check a couple things:

    Make sure you have a valid AddHandler:

    http://httpd.apache.org/docs/current/mod/mod_mime.html#addhandler

    for .cgi, or test that other .cgi's are working in the same directory as extinfo.cgi.

    It looks like you have AllowOverride turned off for this directory, but perhaps the apache configuration allows it for other directories higher up? In that case make sure you haven't disabled ExecCGI or have any .htaccess files in any directory above the nginx sbin.

    If that doesn't work I'd increase the apache log level:

    http://httpd.apache.org/docs/2.0/mod/core.html#loglevel

    LogLevel debug

    then see if that prints anything related to why that file or directory isn't being handled as a CGI.

    If none of that points out the problem I'd probably strace apache after setting it to only run a single child and seeing the output of a working CGI vs a non-working CGI to see what the issue is. This sort of test would expose if maybe there is a configuration issue with the logging so that apache isn't able to write out helpful messages to the CGI log telling us why it refuses to play ball. Hopefully you don't get all the way down to this part before fixing it, if you do I'll be really curious what the diagnostics showed.

    • 0