Home / server / Questions / 548322
Accepted
IttayD
Asked: 2013-10-25 04:11:30 +0800 CST

tool to do statistics in the linux command line

  • 772

say I run a command that outputs space separated values over lines. Some of which are numbers. Is there a utility I can use to calculate the mean, median, standard deviation of these numbers? something like 'cut' but that outputs these statistics.

linux

3 Answers

  1. Best Answer

    Check out st: https://github.com/nferraz/st or sta (recommended): https://github.com/simonccarter/sta

    From GitHub:

    sta

    Simple statistics from the command line interface (CLI), fast. Description

    This is a lightweight, fast tool for calculating basic descriptive statistics from the command line. Inspired by https://github.com/nferraz/st, this project differs in that it is written in C++, allowing for faster computation of statistics given larger non-trivial data sets.

    Additions include the choice of biased vs unbiased estimators and the option to use the compensated variant algorithm.

    Given a file of 1,000,000 ascending numbers, a simple test on a 2.5GHz dual-core MacBook using Bash time showed sta takes less than a second to complete, compared to 14 seconds using st.

    • 1

  2. You might also consider using clistats. It is a highly configurable command line interface tool to compute statistics for a stream of delimited input numbers.

    I/O options

    • Input data can be from a file, standard input, or a pipe
    • Output can be written to a file, standard output, or a pipe
    • Output uses headers that start with "#" to enable piping to gnuplot

    Parsing options

    • Signal, end-of-file, or blank line based detection to stop processing
    • Comment and delimiter character can be set
    • Columns can be filtered out from processing
    • Rows can be filtered out from processing based on numeric constraint
    • Rows can be filtered out from processing based on string constraint
    • Initial header rows can be skipped
    • Fixed number of rows can be processed
    • Duplicate delimiters can be ignored
    • Rows can be reshaped into columns
    • Strictly enforce that only rows of the same size are processed
    • A row containing column titles can be used to title output statistics

    Statistics options

    • Summary statistics (Count, Minimum, Mean, Maximum, Standard deviation)
    • Covariance
    • Correlation
    • Least squares offset
    • Least squares slope
    • Histogram
    • Raw data after filtering

    NOTE: I'm the author.

    • 1

  3. The R language would be perfect for this. It runs in the terminal, its free and probably the most popular statistics languages out there. So, it has tons of documentation on how to do neat things, including the ones you are asking for.

    • 1