playing linux series awk, grep, sed

Learn and live
Living, in order to learn

Awk, grep, sed are the three major tools for linux operation text, and one of the linux commands that must be mastered. The functions of the three are all dealing with text, but the focus is different. Among them, the awk function is the most powerful, but it is also the most complicated. Grep is more suitable for simple search or matching of text, sed is more suitable for editing the matching text, awk is more suitable for formatting text, more complex format processing of text.

All of the following experimental outputs are based on the test file test.log:
20170102 admin,password Open
20170801 nmask,nmask close
20180902 nm4k,test filter


AWK is a language for processing text files. It is a powerful text analysis tool; awk is counted by column, $0 for all columns, $1 for the first column, and $2 for the second column.


  • -F specifies the input file separator, such as -F:
  • -v assigns a user-defined variable, such as -va=1
  • -f reads the awk command from the script file

Note: Only the most commonly used parameters are listed.


Split each column by space and output columns 1 and 4

$ awk '{print $1,$4}' test.log
$ cat test.log | awk '{print $1,$4}'

Custom separator

Use “,” to split, parameter -F

awk -F, '{print $1,$2}' test.log

Use multiple separators, first use space to split, then use the “,” split for the split result

$ awk -F '[ ,]' '{print $1,$2,$3}' test.log #Note that there is a space before the comma

Setting variables

Set awk custom variable with argument -v
Example: Set the variable a to 1

Note: Spaces are required between -v a.

String stitching: (with “” instead of +)

cat test.txt | awk -v a=\" '{print a""$0""a}'

Logic judgment

Output the first column with the record of 20170801

Output the second column is not nmask, nmask record

Built-in variables

NR parameter: output line number

Regular expression

Output all records in the second column that contain the beginning of nm

Output contains records beginning with 2017

Note: there is no ~ here, because no column is specified.

Ignore case {INGORECASE=1}

Match inversion!~

Built-in function

substr string interception

Intercept the first to fourth characters of the first column

split split string

Separate the data in column 2 with a comma and output the contents of column 2 separately.

gsub replacement

Replace nmask in column 2 with nMask


The Linux grep command is used to find strings that match the criteria in the file.


Recursive query

Grep -r nmask /etc/ #View the contents of the /etc directory containing nmask files

Query inversion

grep -v test test.log

but ###
The Linux sed command uses script to process text files.


-e Process the input text file with the script specified in the option.
-f Process the input text file with the script file specified in the option.
-h shows help.
-n displays only the results of the script processing.
-V Displays version information.


a : new, a can be followed by a string, and these strings will appear on the next line
i : insert, i can be followed by a string, and these strings will appear on the previous line
c : Substitute, c can be followed by a string, these strings can replace the line between n1, n2
d : delete
s : Replace, usually this s action can be combined with regular notation! Such as s/old/new/g

Insert operation

Insert a line after the 4th line of the test.log file, the content is nmask

sed -e 4a\nmask test.log

Delete operation

Delete the 2nd and 3rd lines of test.log

cat test.log | sed '2,3d'

Match delete, delete the line with nmask string

Replacement operation

Sed 's/ string to be replaced / new string /g'

Reference article

本文标题:playing linux series awk, grep, sed


发布时间:2017年08月12日 - 10:08

最后更新:2019年08月16日 - 15:08


许可协议: 署名-非商业性使用-禁止演绎 4.0 国际 转载请保留原文链接及作者。

nmask wechat