Home / ubuntu / Questions

Questions[ascii](ubuntu)

Martin Hope
FinleyGibson
Asked: 2020-04-16 04:31:42 +0800 CST
  • 2

I have some data saved in a directory which has a non-ascii character in the name, how can I access the data within?

The directory appears as:

w0p6_t[[0.53.5]?[1.71.2]]

But escaping the special characters using \ before them does not work, nor does surrounding the directory name with quotes as suggested here.

I believe this "?" is a new line character (ascii \012) as ls -lb returns:

w0p7_t[[0.53.5]\n[1.71.2]]

I can see the contents of the directory using the tree command, which shows the directory as:

w0p6_t[[0.53.5]\012[1.71.2]]

along with all of its content.

I have also tried:

cd "w0p7_t[[0.53.5]\n[1.71.2]]"
cd "w0p7_t[[0.53.5]\012[1.71.2]]"

and

cd w1p0_t\[\[0.53.5]\\012\[1.71.2\]\]
cd w1p0_t\[\[0.53.5]\\n\[1.71.2\]\]

with no success.

Also, attempting to auto-complete the command using Tab does not prompt suggestions for this directory.

cd bash directory special-characters ascii
Martin Hope
John
Asked: 2019-07-12 20:08:12 +0800 CST
  • 2

in widows I used to open any txt editor - set Numlock active and while holding Alt using the number pad type 0169 and get the ascii copyright © - How do I get direct ascii access in Linux / Ubuntu Studio Using Mousepad or any txt editor?

ascii
Martin Hope
user364819
Asked: 2015-03-20 11:37:33 +0800 CST
  • 3

I have installed aview the ASCII art image viewer with:

sudo apt-get install aview

But now when I run it with:

aview /path/to/image.extension

I get this:

Invalid magic-not p?m family format

I have tried getting this to work with the following image formats so far, and still the same output: png, jpg, and bmp.

Why is this happening? And how do I fix this, or use aview correctly if I am using it wrong?

OS Information:

Description:    Ubuntu 14.10
Release:    14.10

Package Information:

aview:
  Installed: 1.3.0rc1-9
  Candidate: 1.3.0rc1-9
  Version table:
 *** 1.3.0rc1-9 0
        500 http://gb.archive.ubuntu.com/ubuntu/ utopic/universe amd64 Packages
        100 /var/lib/dpkg/status
ascii
Martin Hope
Peter.O
Asked: 2010-10-30 04:46:19 +0800 CST
  • 5

I have a lot of plain text files which come from a Windows environment.
Many of them use a whacky default Windows code-page, which is neither ASCII (7 bits) nor UTF-8.

gvim has no problem opening these files, but gedit fails to do so.
gvim reports the encoding as latin1.

I assume that gvim is making a "smart" assumption about the code-page.
(I believe this code-page still has international variants).

Some questions arise from this:

  • (1). Is there some way the gedit can be told to recoginze this code-page?
    ** NB. [Update] For this point (1), see my answer, below.
    ** For points (2) and (3). see Oli's answer.

  • (2). Is there a way to scan the file system to identify these problem files?

  • (3). Is there a batch converting tool to convert these files to UTF-8?

(.. this old-world text mayhem was actually the final straw which brought me over to Ubuntu... UTF-8 system-wide by default Brilliant)

[UPDATE]
** NB: ** I now consider the following Update to be partially irrelevent, because the "problem" files aren't the "problem" (see my answer below).
I've left it here, because is may be of some general use to someone.

I've worked out a rough and ready way to identify the problem files...
The file command was not suitable, because it identified my example file as ASCII... but an ASCII file is 100% UTF-8 compliant...

As I mentioned in a comment below, the test for an invalid first byte of a UTF-8 codepoint is:

  • if the first byte (of a UTF-8 codepoint) is between 0x80 and 0xBF (reserved for additional bytes), or greater than 0xF7 ("overlong form"), that is considered an error

I know sed (a bit, via a Win32 port), so I've managed to cobble together a RegEx pattern which finds these offending bytes.

It's an ugly line, so look away now if regular expressions scare you :)

I'd really appreciate it if someone points out how to use hex values in a range [] expression.. I've just used the or operator \| 

fqfn="/my/fully/qualified/filename"  
sed -n "/\x80\|\x81\|\x82\|\x83\|\x84\|\x85\|\x86\|\x87\|\x88\|\x89\|\x8A\|\x8B\|\x8C\|\x8D\|\x8E\|\x8F\|\x90\|\x91\|\x92\|\x93\|\x94\|\x95\|\x96\|\x97\|\x98\|\x99\|\x9A\|\x9B\|\x9C\|\x9D\|\x9E\|\x9F\|\xA0\|\xA1\|\xA2\|\xA3\|\xA4\|\xA5\|\xA6\|\xA7\|\xA8\|\xA9\|\xAA\|\xAB\|\xAC\|\xAD\|\xAE\|\xAF\|\xB0\|\xB1\|\xB2\|\xB3\|\xB4\|\xB5\|\xB6\|\xB7\|\xB8\|\xB9\|\xBA\|\xBB\|\xBC\|\xBD\|\xBE\|\xBF\|\xF8\|\xF9\|\xFA\|\xFB\|\xFC\|\xFD\|\xFE\|\xFF/p" "${fqfn}"

So, I'll now graft this into Oli's batch solution... Thanks Oli!

PS. Here is the invalid UTF-8 byte it found in my sample file ...
"H.Bork, Gøte-borg." ... the "ø" = F8 hex... which is an invalid UTF-8 character.

gedit gvim utf-8 ascii