home How to remove ^[, and all of the escape sequences in a file using linux shell scripting




putty escape codes (9)

ansi2txt command (part of kbtin package) seems to be doing the job perfectly on Ubuntu.

We want to remove ^[, and all of the escape sequences.

sed is not working and is giving us this error:

$ sed 's/^[//g' oldfile > newfile; mv newfile oldfile;
sed: -e expression #1, char 7: unterminated `s' command

$ sed -i '' -e 's/^[//g' somefile
sed: -e expression #1, char 7: unterminated `s' command

You may remove these control characters by:

tr -d "[:cntrl:]" file.txt

however it'll remove also new line endings, so here is a trick, define this alias:

alias clean='tr "\r\n" "\275\276" | tr -d "[:cntrl:]" | tr "\275\276" "\r\n"'

then try like:

cat file.txt | clean > new_file.txt


I found the following solution to successfully parse vt100 color codes and remove the non-printable escape sequences. The code snippet found here successfully removed all codes for me when running a telnet session using telnetlib:

    def __processReadLine(self, line_p):
    '''
    remove non-printable characters from line <line_p>
    return a printable string.
    '''

    line, i, imax = '', 0, len(line_p)
    while i < imax:
        ac = ord(line_p[i])
        if (32<=ac<127) or ac in (9,10): # printable, \t, \n
            line += line_p[i]
        elif ac == 27:                   # remove coded sequences
            i += 1
            while i<imax and line_p[i].lower() not in 'abcdhsujkm':
                i += 1
        elif ac == 8 or (ac==13 and line and line[-1] == ' '): # backspace or EOL spacing
            if line:
                line = line[:-1]
        i += 1

    return line

Python regex to match VT100 escape sequences

The combined expression for escape sequences can be something generic like this:

(\x1b\[|\x9b)[^@-_]*[@-_]|\x1b[@-_]

Should be used with re.I

This incorporates:

  1. Two-byte sequences, i.e. \x1b followed by a character in the range of @ until _.
  2. One-byte CSI, i.e. \x9b as opposed to \x1b + "[".

However, this will not work for sequences that define key mappings or otherwise included strings wrapped in quotes.


Are you looking for ansifilter?


Two things you can do: enter the literal escape (in bash:)

Using keyboard entry:

sed 's/Ctrl-vEsc//g'

alternatively

sed 's/Ctrl-vCtrl-[//g'

Or you can use character escapes:

sed 's/\x1b//g'

or for all control characters:

sed 's/[\x01-\x1F\x7F]//g' # NOTE: zaps TAB character too!


How to replace some colored characters with blank sed

You are replacing it with z just remove the z also instead of 0 seems that you have typed big O. Change it to 0 then it will work very well:

~$ echo "id       ^[[0mevent^[[0m            ^[[0msuite^[[0m"|sed 's/\^\[\[0m//g'
id       event            suite