Recommended: One Linux command per day (3) : sed

Introduction of the command

Awk pattern Scanning and processing language for text and data processing.

Awk is a programming language for text and data processing under Linux/Unix. Data can come from standard input (STDIN), one or more files, or the output of other commands. It is used on the command line, but more as a script. Awk has a lot of built-in functionality, such as arrays, functions, and so on. This is what it has in common with C. Flexibility is awK’s biggest advantage.

Syntax format

awk [options] 'scripts' var=value filename

Copy the code

Commonly used parameters

-f Specifies the delimiter (which can be a string or regular expression). -f Reads the AWK command from the script file. -v var=value Specifies the assignment variable to pass the external variable to AWKCopy the code

Basic script structure

awk 'BEGIN{ print "start" } pattern{ commands } END{ print "end" }' filename

Copy the code

An AWK script usually consists of a BEGIN statement, a pattern matching statement, and an END statement, all of which are optional.

Working principle:

  • The first step is to execute the BEGIN statement

  • The second step reads a line from the file or standard input, and then executes the PATTERN statement, scanning the file line by line until all files are read

  • The third step executes the END statement

Examples show:

echo "hello " | awk 'BEGIN{ print "welcome" } END{ print "2017-08-08" }' welcome 2017-08-08 echo -e "hello" | awk 'the BEGIN {print "welcome"} {print} END {print "2017-08-08"}' welcome hello # 2017-08-08 without the print parameter is the default print only the current line echo | awk '{ a="hello"; b="nihao"; c="mingongge"; print a,b,c; } 'hello nihao mingongge # use print comma-delimited, printed in the blank space boundary echo | awk' {a = "MGG"; b="mingg"; c="mingongge"; print a" is "b" or "c; }' MGG is mingg or mingongge #awk print statementCopy the code

Awk variables

Built-in variables

FS # input field separator (-f) default space RS # input record separator (-f) default space RS # input record separator (-f) Default space ORS # output record separator, default newlineCopy the code

External variables

[mingongge@ ~]#a=100
[mingongge@ ~]#b=100
[mingongge@ ~]#echo |awk '{print v1*v2 }' v1=$a v2=$b
10000

Copy the code

Awk operation and judgment

Arithmetic operator

+ - add and subtract * / & more than beyond ^ * exponentiation + + - to increase or decrease, as a prefix or suffixCopy the code
[mingongge@ ~]#awk 'BEGIN{a="b"; print a,a++,a--,++a; }' b 0 1 1 [mingongge@ ~]#awk 'BEGIN{a="0"; print a,a++,a--,++a; }' 0 0 1 1 [mingongge@ ~]#awk 'BEGIN{a="0"; print a,a++,--a,++a; }' 0 0 0 1 # As in other programming languages, all are used as arithmetic operators to operate, the operands are automatically converted to numeric values, and all non-numeric values are changed to 0Copy the code

The assignment operator

= += -= *= /= %= ^= **=

Copy the code

Regular operator

~! ~ Matches the regular expression or does not match the regular expressionCopy the code

Logical operator

| | && with logic or logicCopy the code

Relational operator

< <= > >= != = 
Copy the code

Other operators

$field references space string linker? : ternary operator ln whether there is a key value in the arrayCopy the code

Awk regular

^ Start of line locator $End of line locator. Matches any single character * Matches zero or more leading characters (including carriage return) + matches one or more leading characters? Match zero or one leading characters in [] to match any one character within a specified character groups / ^ [ab] [^] match is not in any one character within a specified character groups () subexpression | or escape characters ~,! ~ Matching or unmatching conditional statements x{m} x characters repeated m times x{m,} x characters repeated at least m times x{m, n} x characters repeated at least m times but not more than n times (parameter -posix or --re-interval must be specified)Copy the code

This section describes Awk instances

Awk -f: '/^Dan/{print $2}' datafile # '/^[CE]/{print $1}' datafile # print awk -- F: '{if(length($1) == 4) print $1}' awk -- F: Awk -f: '/^Vinh/{print "a"$5}' 2.txt Awk -f: '($5 == 68900) {print $1}' 2. TXT: awk -f: '($5 == 68900) {print $1}' 2. '{if(length($1) == 11) print $1}' 2. TXT awk -f: '$1~/Tommy Savage/ {print $5}' 2. TXT awk -f: '$1~/Tommy Savage/ {print $5}' '($1 = = "Tommy" Savage ") {print $5}' 2. TXT # print with: space and the first listed as Tommy the fifth column content of Savage ll | awk 'BEGIN {size = 0; } {size=size+$5; } END{print "[END]size is ",size}' awk 'BEGIN{size=0; } {size=size+$5; } END{print "[END]size is ",size/1024/1024,"M"}' awk 'BEGIN{a=10; a+=10; Print a} '# 20 a + 10 is equivalent to a = a + 10 echo | awk' BEGIN {a = "100 testaaa"} a ~ / test / {print "ok"} '# if there is a regular matching test characters, Ok awk 'BEGIN{a="b"; print a=="b"?" ok":"err"}' ok awk 'BEGIN{a="b"; print a=="c"?" Ok ":"err"}' err # ternary operator? Awk '/root/{print $0}' passwd # match all lines containing root awk -f: '$5~/root/{print $0}' passwd Line matching the fifth field is the root of the ifconfig eth0 | awk 'BEGIN {FS = "[[: space:]] +} NR = = {print $4}' 2 # print IP address awk '{print toupper ($0)}' test. TXT #toupper is a built-in awK function that converts lowercase letters to uppercaseCopy the code

One Linux command per day (1) : xargs

One Linux command per day (2) : od