Home / server / Questions / 606784
Accepted
Janning
Asked: 2014-06-21 06:43:52 +0800 CST

Recording network traffic peak per second every 5 Minutes

  • 772

I would like to monitor my peak network usages. I already use munin to monitor network traffic in a 5 minute sample. But as we have lot of peaks of http traffic which lasts only a few seconds I want to know the peak network usage per second in the last 5 minutes not the average.

At the moment I use iftop which is an easy to see the peak usage. But iftop can only run interactively. I use screen to get it running and can look into it on a regular basis.

How can I use iftop or a similar tool to just write out the peak network usage in the last 5 minutes every 5 minutes. I just need a file with lines like this:

2014-06-17 15:43:12 TX: 14,3 MBit/s RX: 16,2 MBit/s Sum: 29,6 MBit/s
2014-06-17 15:48:12 TX: 11,3 MBit/s RX: 12,2 MBit/s Sum: 22,3 MBit/s

and so on.

I tried ntop but it is quite a bit of overhead and I can't get the data easily into munin. And it does not write out peaks per 5 Minutes.

Does anybody know a tool or efficient script for this?

linux

4 Answers

  1. Best Answer

    As of Munin 2.0 you can use a feature called supersampling to collect data points with 1-second resolution. The Munin polling interval still will be 5 minutes, the respective Munin plugin is responsible for actively collecting and storing the high-resolution data. There is a sample implementation - the multicpu1sec plugin which is recording mpstat data.

    Now the graphs Munin presents you are a different story. Munin is using RRDtool for graphing, which in turn is able to use different consolidation functions (CF) to represent more than one data point into a single pixel of a graph. As of current, Munin is only able to graph average (AVG) data and there is no configurable support for other consolidation functions (minimum, maximum, total, last).

    You might consider patching Munin accordingly - it does not look too hard to do. Alternatively, you might just invoke rrdtool graph with custom parameters and a different consolidation function as it suits your needs. Just look into your munin-cgi-graph.log to get a starting point for playing around. If you only need it on a case-to-case basis, you also might just zoom into the area of interest (which would need you to have dynamic graphs enabled in your Munin configuration) and look at the Min/Max values printed in the graphs' legend area.

    • 3

  2. Munin is a good tool, and I quite like it. However I have sometimes found that for specific use cases, doing my own sampling, and using rrdtool works better.

    In this case - you can probably get the raw data you want by ifconfig, and look at rx and tx bytes.

    This you can feed into a custom RRD build around DERIVE. 

    rrdtool create netmax.rrd -s 1 DS:tx_bytes:DERIVE:5:0:U DS:rx_bytes:DERIVE:5:0:U RRA:MAX:0.5:1:84600 RRA:AVERAGE:0.5:1:84600 RRA:MAX:0.5:60:12960 RRA:AVERAGE:0.5:60:12960

    Grab the values from ifconfig in a simple script, and update it into rrdtool using:

    rrdtool update netmax.rrd N:12312321:341241

    (e.g. tx bytes, rx bytes from your ifconfig).

    Then graph it:

    rrdtool graph -e now -s end-6h netmax.png DEF:tx_max=netmax.rrd:tx_bytes:MAX DEF:rx_max=netmax.rrd:rx_bytes:MAX LINE1:tx_max#00FF00:tx_max LINE1:rx_max#0000FF:rx_max

    Not too hard to build a script around it for ad-hoc usage, and is exactly what Munin uses behind the scenes.

    • 2

  3. would sar meet your needs? For example 

    sar -n ALL 5 1

    will give you stats for 5 seconds 1 time.

    • 0

  4. Please define "peak".

    Since bandwidth = dataVolume / time, the term is meaningless without reference to the period over which the peak was observed.

    If you want to limit the minimum peak width to 1 second intervals that's fine, but by definition you must therefore sample once every second. And even if you do that, an intense burst of traffic that only lasts 1/10th of a second would appear 10x smaller than it would have if you had sampled at 0.1 second intervals.

    To think of it another way, an analogy for "bandwidth" would be the power of a laser, where power = energy / time. A 1 mW handheld laser pointer is 1 millijoules of (continuous) energy per second and is considered perfectly safe, but if you squeezed that 1 mJ into a single nanosecond you'd have a pulse laser of one megawatt power.

    • 0