Home / server / Questions / 583248
Accepted
zymhan
Asked: 2014-03-20 12:33:05 +0800 CST

Zabbix 2.0 reads logs correctly, but trigger status is "UNKNOWN"

  • 772

I'm trying to monitor a log file for the word "error", and if the word appears, fire a trigger for each new occurence of "error." I'm starting simply, with a file (/var/log/test.log) that is world-readable. It contains the following lines:

test
error
also an error
but not this

The item in Zabbix is log[/var/log/test.log,error], type "Zabbix Agent (Active)", with information type "log".

I'm using the following trigger tied to this item: {nero:log[/var/log/test.log,error].str(error)}=1

I've tried leaving the "error" part out of the item, and changing the trigger value to equal 0. No matter what I do, I get a trigger status of UNKNOWN in the Events tab.

I'm not sure what I'm missing here, does someone see the issue? I checked the agent log on the server, and it doesn't mention any issues.

log-files

4 Answers

  1. Try to use quotation marks in item:

    log[/var/log/test.log,"error"]

    And make sure you use "Zabbix agent (active)" type and "Log" as type of item.

    You might also need to set encoding of file, but I'm not sure on that. I'm neither sure about quotation marks, but documentation says so and many examples I've found says so, when I was working with my problems with logs.

    • 1

  2. First, make sure your item actually gets these error lines. Check Monitoring -> Latest data. If it does not, that's an active agent configuration problem, not log monitoring problem. Verify that the agent hostname (agent daemon configuration file, parameter Hostname) matches the host name in the frontend - note that it is case sensitive.

    If you item had received the error lines, most likely your item got no new values after you set up the trigger. Make sure to mark "Multiple PROBLEM events generation" checkbox for that trigger, then run: $ echo error >> /var/log/test.log

    The agent should pick up the new line and send that to the server. Server evaluates this trigger after getting a new value only.

    • 1
  3. Best Answer

    For the record, I never was able to resolve the issue. Instead, I setup a cron job to loop through each line of the log file, sending it to the Zabbix server with zabbix_sender.

    log_file="/var/log/rsmonit_errors.log"          # File to read
zabbix_server="host.foo.com"       # FQDN or IP of Zabbix Sever
zabbix_host="servername"                             # Host name as defined in the entry in the Zabbix Server. NOT DNS or $HOSTNAME, this is specific to zabbix. Case sensitive.
item_key="rsmonit.errors"                       # Item name, as defined for the host in the Zabbix Server

# Loop through each line and feed it via zabbix_sender to the Zabbix server
while read line; do
    echo $line | xargs -0 zabbix_sender -v -z $zabbix_server -s $zabbix_host -k $item_key -o
done < $log_file

    This seems to work pretty well, though I do seem to get some duplicate alerts, instead of unique alerts for each line. Please feel free to point out any places this could be improved, as this is my first time making a script like this.

    • 0

  4. Zabbix, like many other modern monitoring applications allows its users to implement custom plugins. In your case, to monitor log files as painlessly as possible, you will need a plugin that was built specifically for that; one that allows you to easily specify the log patterns to look for and the thresholds to assign to those patterns when they're found in the log.

    An example of such a tool is autoresolve.kl.sh

    The installation process for this tool is simple:

    1. Log into the host on which you have log files to monitor
2. wget (the-url-link-of-zip-file-of-autoresolve.kl.sh)
3. cd /tmp ; unzip (the-downloaded-zip-file)
4. ./install.sh /var/tmp/KINGLAZY/SHIELDX-autoresolve.kl.sh /home/jserver -force

    Replace '/home/jserver' with your zabbix plugins directory. Also, make sure to run the preceding installation commands as an ordinary user - not root, unless you're testing.

    After installation, you can begin monitoring logs with a simple command such as this:

    ./autoresolve.kl.sh localhost /var/tmp/logXray,fixer,0n-1y-2y,0-uname,1-who,2-uptime autonda /var/log/rsmonit_errors.log 60m 'error' '.' 1 2 error_watch -ndshow

    The most important parameters you'll need to change are:

    1. /var/log/rsmonit_errors.log - This is the log file you're monitoring

    2. error - This is the string you're looking for in the log file. To monitor multiple patterns inside a log, just specify them this way: "error_P_fail_P_segfault_P_crash_P_panic". The underscore P underscore means "|". It's a safer way to specify the pipe symbol when using the tool with external applications...such as Zabbix.

    3. error_watch - This is the name/tag that you're assigning to this particular log check. Later, if you wish, this tag can be used to help you generate graphs on the monitored log.

    For a detailed explanation on what each parameter means, you can visit the help page directly.

    On the same help page, you'll also find step by step instructions on how to get this tool to work with Zabbix...i.e. what zabbix configuration file you need to update...and what settings you need to have on the zabbix web interface.

    • -2