Managing Text Files

Viewing file contents

Use less %99.9 of the time

less {file}

{command} | less

Display last 10 lines with tail

tail -n 10 {file}

Display first 10 lines with head

head -n 10 {file}

Display 10th line from the bottom with head and tail

tail -n 10 {file} | head -n 1

cut, sort, tr

Extract the first field delimited by ’:’ from a file

cut -d ':' -f 1 /etc/passwd

Extract the first field delimited by ’:’ from a file and sort

cut -d ':' -f 1 /etc/passwd | sort

Sort numerically (-n) based on the third field/key (-k3) delimited by ’:’ (-t :)

sort -t : -k3n /etc/passwd

Translate tr lowercase to uppercase

echo hello | tr [:lower:][:upper:]

Using grep

Show matches and include 5 lines before

ps faux | grep -B5 bash

Show matches and include 5 lines after

ps faux | grep -A5 bash

Show matches and include 5 lines before and after

ps faux | grep -C5 bash

Search in all files in a directory

grep cjordan /etc/*

Search in all files in a directory and output files, not lines

grep -l cjordan /etc/*

Case insensitive search in all files in a directory

grep -i cjordan /etc/*

Recursive search and follow all symbolic links

grep -R cjordan /etc

Recursive search and follow all symbolic links only if they are on the command line

grep -r cjordan /etc

Prefix results with line number

grep -n cjordan /etc/*

Applying Regular Expressions

Regex man page

man 7 regex

• ^ beginning of the line: grep '^|' myfile
• $ end of the line: grep 'anna$' myfile
• \b end of word: grep '^lea\b' myfile will find lines starting with lea, but not with leanne
• . one character: grep '^.$' myfile will find lines with one of any character
• * zero or more times: grep 'n.*x' myfile will match any amount of characters between n and x
• + one or more times (extended): grep -E 'bi+t' myfile will match bit, biit, biiit, etc, but not bt
• ? zero or one time (extended): grep -E 'bi?t' myfile will match bt or bit, but not biit
• \n{3\} n occurs 3 times: grep 'bon\{3\}nen' myfile will match bonnnnen
• string must be a word: grep '\banna\b' myfile
• match on either option (extended): grep -E 'foo|bar' myfile

Exploring awk

Extract the fourth field delimited by a colon

awk -F : '{ print $4 }' /etc/passwd

Extract the fourth field delimited by a colon of a line that contains cjordan

awk -F : '/cjordan/ { print $4 }' /etc/passwd

Extract the last field delimited by a colon that contains cjordan

awk -F : '/cjordan/ { print $NF }' /etc/passwd

Using sed

Print the 5th line

sed -n 5p {file}

Search and replace

sed 's|anna|otto|g' {file}

Search and replace modifying file in place

sed -i 's|anna|otto|g' {file}

Delete second line from file

sed -i -e '2d' {file}

Insert line after match (a\ append)

sed -i "/^foo/a\bar" {file}

Insert line before match (i\ insert)

sed -i "/^foo/i\bar" {file}

Lab Exercise

• First, create a file, ~/stuff with the following contents:

foo
bar
bing
bang
sally
susan
jeremy

1. Use head and tail to display the 5th line of the file /etc/passwd
2. Use sed to display the 5th line of the file /etc/passwd
3. Use sed to replace ‘foo’ with ‘fooski’ in ~/stuff. Only output the results to stdout.
4. Use sed to insert ‘bob’ on a separate line above ‘bing’ in ~/stuff. Modify the file in place
5. Use sed to insert ‘joe’ on a separate line below ‘bob’ in ~/stuff. Modify the file in place
6. Use awk to filter the last column out of ps aux
7. Use grep to show the names of all files in /etc that have lines that contain ‘root’ as a word
8. Use grep to show all lines from all files in /etc that contain exactly 3 characters

Solution

1. head -n 5 /etc/passwd | tail -n 1
2. sed -n 5p /etc/passwd
3. sed "s|foo|fooski| stuff
4. sed -i "/bing/i\bob" stuff
5. sed -i "/bob/a\joe" stuff
6. ps aus | awk '{print $NF}'
7. grep -rwl root /etc
8. grep -rx '...' /etc