1. Technology
You can opt-out at any time. Please refer to our privacy policy for contact information.

Discuss in my forum

GNU/Linux Command-Line Tools Summary - Text manipulation tools



  • uniq

    Eliminates duplicate entries from a file and it sometimes greatly simplifies the display.

    uniq options:

    • -c --- count the number of occurances of each duplicate

    • -u --- list only unique entries

    • -d --- list only duplicate entries

    For example:


    uniq -cd phone_list.txt

    This would display any duplicate entries only and a count of the number of times that entry has appeared.

  • tr

    (translation). A filter useful to replace all instances of characters in a text file or "squeeze" the whitespace.



    cat some_file | tr '3' '5' > new_file

    This will run the cat program on some file, the output of this command will be sent to the tr command, tr will replace all the instances of 3 with 5, like a search and replace. You can also do other things such as:


    cat some_file | tr '[A-Z]' '[a-z]' > new_file

    This will run cat on some_file and convert any capital letters to lowercase letters (you could use this to change the case of file names too...).


    You can also do a search and replace with a one line Perl command, read about it at the end of this section.

  • nl

    The number lines tool, it's default action is to write it's input (either the file names given as an argument, or the standard input) to the standard output.

    Line numbers are added to every line and the text is indented.

    This command can do take some more advanced numbering options, simply read the info page on it.

    These advanced options mainly relate to customisation of the numbering, including different forms of separation for sections/pages/footers etc.

    Also try cat -n (number all lines) or cat -b (number all non-blank lines). For more info on cat check under this section: Section 11.2

    There are two ways you can use nl :


    nl some_text_file.txt

    The above command would add numbers to each line of some_text_file. You could use nl to number the output of something as shown in the example below;


    grep some_string some_file | nl

  • Perl search and replace text

    To search and replace text in a file is to use the following one-line Perl commandadvanced bash scripting guide :


    $ perl -pi -e "s/oldstring/newstring/g;" filespec [RET]

    In this example, "oldstring" is the string to search, "newstring" is the string to replace it with, and "filespec " is the name of the file or files to work on. You can use this for more than one file.

    Example: To replace the string "helpless" with the string "helpful" in all files in the current directory, type:


    $ perl -pi -e "s/helpless/helpful/g;" * [RET]

    Also try using tr to do the same thing (see further above in this section).

           If these tools are too primitive

    If these text tools are too simple for your purposes then you are probably looking at doing some programming or scripting.

    If you would like more information on bash scripting then please see the advanced bash scripting guide , authored by Mendel Cooper.


    1. About.com
    2. Technology
    3. Linux

    ©2014 About.com. All rights reserved.