I have such a file with multiple blank lines in vscode
$ tail -n 20 draft3.py
hi = len(a)
if lo < 0:
raise ValueError('low must be non-negative')
if lo == hi:
return None
mid = (lo + hi) // 2
if x == a[mid]:
return x
if x > a[mid]:
lo = mid + 1
return self.bi_search(a, x, lo, hi)
if x < a[mid]:
hi = mid
return self.bi_search(a, x, lo, hi)
Tried multiple methods to remove the blank lines
`grep -v -e '^$' failed
$ tail -n 20 draft3.py | grep -v -e '^$'
hi = len(a)
if lo < 0:
raise ValueError('low must be non-negative')
if lo == hi:
return None
mid = (lo + hi) // 2
if x == a[mid]:
return x
if x > a[mid]:
lo = mid + 1
return self.bi_search(a, x, lo, hi)
if x < a[mid]:
hi = mid
return self.bi_search(a, x, lo, hi)
`grep -Ev "^$" failed
$ tail -n 20 draft3.py | grep -Ev "^$"
hi = len(a)
if lo < 0:
raise ValueError('low must be non-negative')
if lo == hi:
return None
mid = (lo + hi) // 2
if x == a[mid]:
return x
if x > a[mid]:
lo = mid + 1
return self.bi_search(a, x, lo, hi)
if x < a[mid]:
hi = mid
return self.bi_search(a, x, lo, hi)
`sed '/^$/d' failed
$ tail -n 20 draft3.py | sed '/^$/d'
hi = len(a)
if lo < 0:
raise ValueError('low must be non-negative')
if lo == hi:
return None
mid = (lo + hi) // 2
if x == a[mid]:
return x
if x > a[mid]:
lo = mid + 1
return self.bi_search(a, x, lo, hi)
if x < a[mid]:
hi = mid
return self.bi_search(a, x, lo, hi)
What's the problem? How could remove the blank lines?
Presumably you want to remove not only empty lines, but also lines with only whitespace characters. For that, use:
The
sed
expressiond
eletes every line with any number (*
) of whitespace characters (\s
) in it.grep -v
outputs any line which does not match the expression.Example usage
grep -v '^$'
will remove empty lines. But what if we have spaces or tabs in some lines ? For example I added 3 spaces to parts of your text, and if we docat -A
we will see that it shows line terminator$
, but it will be offset.The second line there has 3 spaces, first one doesn't. So we also want to use
[[:blank:]]
character class to account for those as well:Now you should see that the line with 3 added spaces is gone. The
*
signifies zero or more repetitions of the characters, so the pattern^[[:blank:]]*$
also implies^$
when there are zero whitespace or tab characters on the line. So this pattern handles both truly empty and seemingly empty lines. It also applies exactly the same togrep
orsed
, because we're using basic regex expressions and[[:blank:]]
is one of the POSIX character classes, so it is portable.We could also do something like this in python but without regex patterns:
Why does this work ? Because
.split()
on a string will split at whitespaces to extract non-whitespace tokens. If a line contains only spaces, the resulting list from.split()
will be empty.As noted by ilkkachu in the comments, the issue can also occur if you use CRLF line endings ( used in DOS/Windows text files). It is easy to see if the file uses CRLF line endings via
cat -A
, they will be marked as^M
. For example,One thing that could be done to account for carriage return is this:
It may be simpler to first use a
dos2unix
utility designed specifically for converting DOS files to Unix files, and then usesed
andgrep
. See ByteCommander's answer that shows example of how to do that.use
and you will have
Only with sed is
This grep command works and even accounts for dos carriage returns, blank/empty lines, and blank lines that contain empty spaces and/or tabs: