Without uniq:
amin@ubuntu:~/Desktop$ cut -f 1 info.log | tail -n +2 | head -n -1 | sort
Abol
Abol
Ahmad
Akbar
Arash
Hadi
Hamed
Mahmood
Maryam
Maryam
Mohsen
NIma
Rasool
Sadegh
Sepide
Sepide
With uniq:
amin@ubuntu:~/Desktop$ cut -f 1 info.log | tail -n +2 | head -n -1 | sort | uniq
Abol
Abol
Ahmad
Akbar
Arash
Hadi
Hamed
Mahmood
Maryam
Mohsen
NIma
Rasool
Sadegh
Sepide
Sepide
As you see result are same in both, why?
I have a log file sorted by IP addresses, I want to find the number of occurrences of each unique IP address. How can I do this with bash? Possibly listing the number of occurrences next to an ip, such as:
5.135.134.16 count: 5
13.57.220.172: count 30
18.206.226 count:2
and so on.
Here’s a sample of the log:
5.135.134.16 - - [23/Mar/2019:08:42:54 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
5.135.134.16 - - [23/Mar/2019:08:42:55 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
5.135.134.16 - - [23/Mar/2019:08:42:55 -0400] "POST /wp-login.php HTTP/1.1" 200 3836 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
5.135.134.16 - - [23/Mar/2019:08:42:55 -0400] "POST /wp-login.php HTTP/1.1" 200 3988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
5.135.134.16 - - [23/Mar/2019:08:42:56 -0400] "POST /xmlrpc.php HTTP/1.1" 200 413 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:05 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:06 -0400] "POST /wp-login.php HTTP/1.1" 200 3985 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:07 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:08 -0400] "POST /wp-login.php HTTP/1.1" 200 3833 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:09 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:11 -0400] "POST /wp-login.php HTTP/1.1" 200 3836 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:12 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:15 -0400] "POST /wp-login.php HTTP/1.1" 200 3837 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:17 -0400] "POST /xmlrpc.php HTTP/1.1" 200 413 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.233.99 - - [23/Mar/2019:04:17:45 -0400] "GET / HTTP/1.1" 200 25160 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36"
18.206.226.75 - - [23/Mar/2019:21:58:07 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "https://www.google.com/url?3a622303df89920683e4421b2cf28977" "Mozilla/5.0 (Windows NT 6.2; rv:33.0) Gecko/20100101 Firefox/33.0"
18.206.226.75 - - [23/Mar/2019:21:58:07 -0400] "POST /wp-login.php HTTP/1.1" 200 3988 "https://www.google.com/url?3a622303df89920683e4421b2cf28977" "Mozilla/5.0 (Windows NT 6.2; rv:33.0) Gecko/20100101 Firefox/33.0"
18.213.10.181 - - [23/Mar/2019:14:45:42 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
18.213.10.181 - - [23/Mar/2019:14:45:42 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
18.213.10.181 - - [23/Mar/2019:14:45:42 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
I have this code in a shell script:
sort input | uniq -c | sort -nr > output
The input file had no preceding white spaces, but the output does. How do I fix this? This is in bash
I have one xlsx file (110725x9 matrix) and I saved as type text (tab delemited) because I don't know whether Unix helps for xlsx files or not. Duplicates rows are always successive line by line.
For example, suppose text file as follow. You will see 3,4-th, 7,8-th and 17,18-th rows are same. I'd like to remove upper duplicate lines not lower always.
2009,37214611872 2009 135 20 17,1 17,4 19,2 21,8 24,1
2009,37237442922 2009 135 22 16,5 14,5 12,6 11,2 10,5
2009,37260273973 2009 136 0 7,7 7,2 7,1 7,3 7,5
2009,37260273973 2009 136 0 7,7 7,2 7,0 7,2 7,4
2009,37488584475 2009 136 20 14,6 15,1 16,4 18,3 20,1
2009,37511415525 2009 136 22 15,9 14,6 12,8 10,9 9,4
2009,37534246575 2009 137 0 8,2 6,9 6,2 6,2 6,4
2009,37534246575 2009 137 0 8,1 6,8 6,1 6,0 6,3
2009,37557077626 2009 137 2 6,8 6,7 6,5 6,3 6,2
2009,37579908676 2009 137 4 5,8 5,6 5,4 5,4 5,7
2009,37602739726 2009 137 6 6,3 6,1 5,9 5,8 5,8
2009,37625570776 2009 137 8 4,5 5,2 6,0 6,6 7,2
2009,37648401826 2009 137 10 9,6 9,0 8,4 8,4 9,1
2009,37671232877 2009 137 12 11,4 11,7 12,4 13,4 14,4
2009,37694063927 2009 137 14 12,4 13,1 14,2 15,4 16,7
2009,37785388128 2009 137 22 15,5 14,0 12,2 10,3 8,7
2009,37808219178 2009 138 0 6,3 5,8 5,5 5,5 5,8
2009,37808219178 2009 138 0 6,2 5,7 5, 4 5,4 5,7
So output should be like that:
2009,37214611872 2009 135 20 17,1 17,4 19,2 21,8 24,1
2009,37237442922 2009 135 22 16,5 14,5 12,6 11,2 10,5
2009,37260273973 2009 136 0 7,7 7,2 7,0 7,2 7,4
2009,37488584475 2009 136 20 14,6 15,1 16,4 18,3 20,1
2009,37511415525 2009 136 22 15,9 14,6 12,8 10,9 9,4
2009,37534246575 2009 137 0 8,1 6,8 6,1 6,0 6,3
2009,37557077626 2009 137 2 6,8 6,7 6,5 6,3 6,2
2009,37579908676 2009 137 4 5,8 5,6 5,4 5,4 5,7
2009,37602739726 2009 137 6 6,3 6,1 5,9 5,8 5,8
2009,37625570776 2009 137 8 4,5 5,2 6,0 6,6 7,2
2009,37648401826 2009 137 10 9,6 9,0 8,4 8,4 9,1
2009,37671232877 2009 137 12 11,4 11,7 12,4 13,4 14,4
2009,37694063927 2009 137 14 12,4 13,1 14,2 15,4 16,7
2009,37785388128 2009 137 22 15,5 14,0 12,2 10,3 8,7
2009,37808219178 2009 138 0 6,2 5,7 5, 4 5,4 5,7
How can I do that without sorting?
From the document said that the option --count going to
prefix lines by the number of occurrences
but after I do some test it seems doesn't work.
$ echo -e "a\na\b\nc" |uniq --count
1 a
1 a
1 c
Should the line 'a' occurs 2 times?