Home / server / Questions / 532183
In Process
tudor -Reinstate Monica-
Asked: 2013-08-20 16:59:15 +0800 CST

Is there a tool that can top-ify an input stream?

  • 772

I come across quite a few use cases where it would be very useful to take input from a (usually newline-delimited) stream and summarise it in a top-like fashion (see top, iotop, etc). A kind of on-the-fly pivot table.

e.g. Take the log-esque input:

I heard A from unit 1 and it said "Great!" 56
I heard A from unit 2 and it said "Oh no!" 42
I heard C from unit 1 and it said "Waiting for input." 33
I heard B from unit 3 and it said "Stopped." -1
...

From this, we could run a tool with a regex and group indicators:

topify [lineout] [regex] [name #1] [group #1] [name #2] [group #2] [All other columns name position]
     where:
         lineout is the number of lines before removing it from the display
         regex is a regex of the lines to match, complete with group indicators
         name #n is a string for the title of column n
         group #n is the number of the group in the regex

e.g.

topify '/^I heard ([A-Z]) from unit ([1-9]) and it said "(.*)" ([-0-9]*)$/' Unit 2 Status 1 Message 3 RetVal 4

This would display in an interactive way, such that the columns could be selected/reordered, etc.:

Unit Status Message            Retval
1    C      Waiting for input. 33
2    A      Oh no!             42
3    B      Stopped.           -1

I understand the brittleness of it, but I'd be really surprised if it hadn't been built before and wanted to check before I go into building it. I also appreciate it isn't particularly complicated to write so maybe everyone has just implemented their own solution...

Has anyone seen a tool like this?

(Please excuse the tags I've used here. I know I may be pushing/breaking the rules of some tags, but this is very generalised. Suggestions welcome.)

logging

1 Answers

  1. You don't need to write a tool, the standard unix toolset can accommodate you just fine.

    #!/bin/bash
echo -e 'Unit\tStatus\tMessage\t\t\tRetval'
cat /var/log/filename | awk '{match($0,"\".*\"",a)}{print $6 "\t" $3 "\t" a[0] "\t\t" $NF}' |sort -k<fieldnum>

    Put that in a .sh file and run watch on it.

    • 1