Home / server / Questions / 210982
Accepted
berkes
Asked: 2010-12-10 11:08:51 +0800 CST

Count amount of new lines with interval in log

  • 772

Is there a way/tool to tail or watch a logfile and report the amount of lines it grows?

E.g. I want to see the amount of new entries in mysql.log every 3 seconds.

watch wc -l /path/to/log.log1 shows only acumulated amount. I'd prefer not to truncate the log inbetween.

logging tail watch

4 Answers

  1. Best Answer

    All you have to do is redirect the file into wc and it will only show what was added rather than the total.

    watch wc -l < /path/to/log.log1

    The < makes all the difference.

    • 1

  2. Here's some quick-and-dirty perl to do what you want:

    #!/usr/bin/perl

open(I, "<$ARGV[0]") || die "Can't open $ARGV[0]: $!\n";

while (1) {
  my $count = 0;
  while (<I>) { $count++; }
  print scalar(localtime()), ": $count lines\n";
  sleep(3);
  seek(I, 0, 1); # clear the EOF on I
}

close(I);

    Here's the output of it running in one window:

    [root@g3 tmp]# perl tt.pl FILE
Thu Dec  9 13:18:38 2010: 0 lines
Thu Dec  9 13:18:41 2010: 0 lines
Thu Dec  9 13:18:44 2010: 0 lines
Thu Dec  9 13:18:47 2010: 0 lines
Thu Dec  9 13:18:50 2010: 0 lines
Thu Dec  9 13:18:53 2010: 1 lines
Thu Dec  9 13:18:56 2010: 0 lines
Thu Dec  9 13:18:59 2010: 1 lines
Thu Dec  9 13:19:02 2010: 0 lines
Thu Dec  9 13:19:05 2010: 0 lines
Thu Dec  9 13:19:08 2010: 0 lines
Thu Dec  9 13:19:11 2010: 0 lines
Thu Dec  9 13:19:14 2010: 0 lines
Thu Dec  9 13:19:17 2010: 0 lines
Thu Dec  9 13:19:20 2010: 100 lines

    And here's the commands I ran in another window to generate the output above:

    [root@g3 ~]# cd /tmp
[root@g3 tmp]# echo foo > FILE
[root@g3 tmp]# echo foo >> FILE
[root@g3 tmp]# for i in `seq 1 100`
> do
> echo $i >> FILE
> done
[root@g3 tmp]#

    Hope that helps...

    • 0

  3. I don't know any standard tool but you could use the following Python script:

    import sys, time

if not len(sys.argv) >= 2:
    print 'usage: %s FILENAME FREQUENCY' % (sys.argv[0])
    sys.exit(1)

fd=open(sys.argv[1])
freq=int(sys.argv[2])
fd.seek(0, 2)
while True:
    current = fd.tell()
    nlines = len(fd.readlines())
    if nlines:
        print nlines
    time.sleep(freq)

    To test:

    $ (while true; do echo test; sleep 0.2; done) > log &
$ python watchandcount.py log 3

    It is not perfect because it cannot "monitor" STDIN or detect when the file has been replaced (the --follow=name tail's option.

    • 0

  4. And here is a bash script:

    TOTAL=0
while true; do
  NEW_TOTAL=$(wc -l /path/to/log.log1 | cut -f1 -d' ')
  echo $(($NEW_TOTAL - $TOTAL))
  TOTAL=$NEW_TOTAL
  sleep 3
done

    It has the advantage of being usable directly in the shell, provided you are not alergic to ";" ;)

    TOTAL=0; while true; do NEW_TOTAL=$(wc -l TOTAL=0; while true; do NEW_TOTAL=$(wc -l big | cut -f1 -d' '); echo $(($NEW_TOTAL - $TOTAL)); TOTAL=$NEW_TOTAL; sleep 3; done | cut -f1 -d' '); echo $(($NEW_TOTAL - $TOTAL)); TOTAL=$NEW_TOTAL; sleep 3; done
    • 0