Home / server / Questions / 252686
Accepted
cherouvim
Asked: 2011-03-29 04:10:36 +0800 CST

"group by count" for IPs in text file in linux

  • 772

I've got a huge textfile which contains IP addresses:

123.33.22.33
221.23.128.2
123.33.22.33
92.222.192.12
92.222.192.12
123.33.22.33

I can sort it to:

123.33.22.33
123.33.22.33
123.33.22.33
221.23.128.2
92.222.192.12
92.222.192.12

and see (with bare eye) that first IP occurs three times, second once, and last one twice.

I'd like to be able to do this in huge logfiles, obviously in an automated way. Is it possible?

thanks

linux analysis text

3 Answers

  1. Best Answer
    cat file_with_ips | sort -nr | uniq -c | sort -nr -k 1

    will sort desc by ip counts, showing the counters on first column

    e.g.

    root@pinkpony:~# cat /tmp/xx
123.33.22.33
221.23.128.2
123.33.22.33
92.222.192.12
92.222.192.12
123.33.22.33
root@pinkpony:~# cat /tmp/xx | sort -nr | uniq -c | sort -nr -k1
      3 123.33.22.33
      2 92.222.192.12
      1 221.23.128.2
root@pinkpony:~#
    • 9

  2. Once you have it sorted you can pipe it throu uniq -c

    cat sorted.list | uniq -c

  2 1.2.3.4
  1 5.6.7.8

    That'll tally the counts for you.

    • 2

  3. You can use uniq -c to get a count, but be sure to sort the input first or you'll only get counts of consecutive, alike lines:

    sort yourfile | uniq -c

    I often then sort it by the lines with the most matches:

    sort yourfile | uniq -c | sort -nr
    • 2