Home / server / Questions / 232838
Accepted
warren
Asked: 2011-02-09 10:22:04 +0800 CST

`tail -f` sometimes stops updating - and the file hasn't moved

  • 772

I have noticed recently that sometimes tail -f <logfile> will stop updating to the screen.

Doing a Ctrl>-C and restarting the tail works fine, though. And I checked to make sure the logfile isn't being rotated midstream (which can make tail lose its mind).

What would cause this? I'm running RHEL 5.2 x64.

logging rhel5 tail

4 Answers

  1. Try using:

    tail --follow=name <logfile>

    And see if that works better. You don't have to worry about it being rotated out from under you.

    Any pattern to it stopping? Certain length of time? Certain time of day?

    • 10
  2. Best Answer

    Try wrapping your tail command with strace if you have it:

    strace -Tt -o /tmp/tail.trace tail -f /var/log/messages

    Then just for crazy recursive kicks you can tail the strace output (doesnt' matter if that breaks because its going out to a file):

     tail -f /tmp/tail.trace

    Mine looks like:

    8:39:00 write(1, "ng SMAC\n", 8)       = 8 <0.000026>
18:39:00 read(3, "", 0)                 = 0 <0.000019>
18:39:00 fstat64(3, {st_mode=S_IFREG|0640, st_size=92990, ...}) = 0 <0.000019>
18:39:00 fstatfs64(3, 84, {f_type="EXT2_SUPER_MAGIC", f_bsize=4096, f_blocks=4807069, f_bfree=1924458, f_bavail=1680271, f_files=1221600, f_ffree=820806, f_fsid={-1331083162, -1313908385}, f_namelen=255, f_frsize=4096}) = 0 <0.000021>
18:39:00 inotify_init()                 = 4 <0.000033>
18:39:00 inotify_add_watch(4, "/var/log/messages", IN_MODIFY|IN_ATTRIB|IN_DELETE_SELF|IN_MOVE_SELF) = 1 <0.000041>
18:39:00 fstat64(3, {st_mode=S_IFREG|0640, st_size=92990, ...}) = 0 <0.000019>
18:39:00 read(4,

    The -t switches on the time and -T switches on time spent in calls.

    Hit return 4 or 5 times to make a bit of vertical space, then wait for it to stop tailing. Hopefully there will be some clues in the output.

    • 6

  3. Given that both problematic logfiles are written by different components of the same application, I wonder if it's not some part of the logging code for that application that's causing the problem. I propose two tests to get a better idea of what's going on:

    • Note the inode of the logfile (ls -i logfile) prior to starting the tail, and once the tail fails, check it again. If the inode has changed, then the logger is rewriting the entire logfile, which would break tail's connection.

    • Note the last line before tail stops working, and then visit the file and find the first log entry after that line. Do this 3-5 times if possible. If it's a problem with the program doing the logging, the part of the program that wrote the log entry immediately after you see tail break is most likely responsible. If that log entry is always the same, or if it comes from the same component of the program, you may have enough data to submit a problem report to the application vendor.

    Good luck.

    • 4

  4. Having the same problem here.

    Problem was, that the file i'm watching was mounted from a different machine. The change notify was not propagated through the mount.

    Solution was to use tail on the original machine.

    • 3