Home / server / Questions / 836001
In Process
drewrockshard
Asked: 2017-03-03 19:05:18 +0800 CST

BASH - sed replace IP variants

  • 772

It's been awhile since I've posted a question on here. Need some assisteancew getting my sed to work. I've tried all kinds of forms of sed options, but can't seem to get this working. I have a file that has "192.*" all over it, and need any type of IP address that has any resemblence of "192" in it, all deleted, but I need to literally keep everything else. Something that looks like:

+ : root : 192.,local,10.
+ : root : local,192.
+ : root : local,192.,10.
+ : root : local,10.33.45.34,192.168.22.100
+ : root : local,192.168.22.100,10.33.45.34
+ : root : local,192.168
+ : root : local,192.168,10.

Item's I've tried:

sed -e 's/,192.*$//g' -e 's/,192.*,//g' -e 's/192.,//g'
sed -e 's/,192.*$//' -e 's/,192.*,//' -e 's/192.,//' 
sed -e 's/,192.*$//' -e 's/,192.*,//' -e 's/192.*,//' 
sed -e 's/,192.*$//' -e 's/,192.*,//' -e 's/192.,//'

I've just about gone crazy, so some help is appreciated.

Thanks in advance.

linux redhat bash sed

2 Answers

  1. I've checked this sed and it is nearly working

    sed -e 's/,\?192\(\.\([0-9]\{0,3\}\)\)\{0,3\},\?//g'

    As example:

    [batchhercules@hercules ~]$ cat example.txt | sed -e 's/,\?192\(\.\([0-9]\{0,3\}\)\)\{0,3\},\?//g'
+ : root : local,10.
+ : root : local
+ : root : local10.
+ : root : local,10.33.45.34
+ : root : local10.33.45.34
+ : root : local
+ : root : local10.

    As you can see on line 3,5,7 it is replacing two commas, which is bad. So upgraded with:

    sed -e 's/,192\(\.\([0-9]\{0,3\}\)\)\{0,3\}//g' -e 's/192\(\.\([0-9]\{0,3\}\)\)\{0,3\},//g'

    Which for your example will work pretty fine:

    [batchercules@hercules ~]$ cat example.txt | sed -e 's/,192\(\.\([0-9]\{0,3\}\)\)\{0,3\}//g' -e 's/192\(\.\([0-9]\{0,3\}\)\)\{0,3\},//g'
+ : root : local,10.
+ : root : local,10.134.
+ : root : local,10.
+ : root : local,10.33.45.34
+ : root : local,10.33.45.34
+ : root : local
+ : root : local,10.
    • 2

  2. My suggestion: Use Perl.

    Where perl-test is a file containing the data in the OP:

    $ perl -pe 's/(?:(\s)192.*?(?:,|$))|(?:,192.*?(,|$))/$1$2/g' perl-test 
+ : root : local,10.
+ : root : local
+ : root : local,10.
+ : root : local,10.33.45.34
+ : root : local,10.33.45.34
+ : root : local
+ : root : local,10.

    You can just as easily cat the contents and pipe it into the perl -pe....

    Perl's regular expression parsing is much like sed, but more powerful.

    Explanation:

    (?:) -- Prepending the contents of the parentheses with ?: makes it a non-capturing group. This way, the parentheses you use only for grouping don't affect the back-reference $1 and $2 variables you use in your replacement string.

    (\s) -- Captures whitespace to be used as $1 in the replacement string. You want to match a 192 IP address preceded by whitespace so that you can recognize it as the start of the list and keep the comma that follows it. You still want to keep the whitespace.

    192.*? -- Non-greedy match. Matches up to the NEXT comma. Default behavior in Perl (and the ONLY behavior in sed) is to match up to the LAST comma. This is not what you want, and you would see a difference in lists with more than three IP addresses.

    (?:,|$) -- Non-capturing group telling the non-greedy match to end at the first comma, or at end-of-line.

    | -- Obviously, alternates to the next regex to match.

    ,?192.*? -- Non-greedy match for 192 that does NOT come at the beginning of a list.

    (,|$) -- Just like earlier, match the first comma following the 192 IP address. In this case, you DO want to capture the comma. You want to keep it since you got rid of the preceding comma with ,?192.*? and there may be another item in the list following it. Of course, if there's NOT another item in the list following it, you'll match the end-of-line $. Referencing $ as $2 in the replacement string simply won't do anything; referencing , will prepare you for the next list item. So you're covered either way.

    Hope this makes some kind of sense!

    • 1