​

1. The awk command

AWK is a language for processing text files. It is a powerful text analysis tool. It is a data processing engine that checks input text based on pattern matching, processes it line by line and outputs it. Usually used in Shell scripts to get specified data, used alone to make statistics on text data.

1. Awk syntax format

  • Format: 1 front command | awk [option] {editing instructions} ‘conditions’ file
  • Format 2: AWk [options] ‘condition {edit instruction}’ file
  • Format 3: awk [Options] ‘BEGIN{edit instruction} {Edit instruction} END{edit instruction}’ file

2, common options

-f: Specifies the delimiter. Multiple delimiters can be specified and can be omitted (default space or Tab bit).

-v: calls the external variable variable

3. Awk built-in variables

FS: Saves or sets the field delimiter, such as FS= “:”

$n: specifies the NTH delimited field, for example, 3 for columns 1 and 3, respectively

$0: The entire line of text currently read

NF: Records the number of fields in the current row (column number)

NR: Records the current number of rows read (line number)

FNR: indicates the number of the current line in the source file

4, case

Contents of a.log are as follows:

aaaaaaaaaa,bbbbbbbbbbbb,cccccccaaaaaaaaaa,bbbbbbbbbbbb,cccccccaaaaaaaaaa,bbbbbbbbbbbb,ccccccc

  • Extract the a.log file separated by (,) in the first column
Awk -f "," '{print $1}' a.logCopy the code

Contents of B. Log are as follows:

I am mike,my mobile is 88848

  • If I want to take Mike and 88848
Awk -f '[,]+' '{print $3" "$7}' b.log #Copy the code

  • Look at lines 2 through 3 of the c.log file
awk '{if(NR>=2 && NR<=3) print $0}' c.log
Copy the code

  • Count the number of users
awk '{count++; print $0; } END{print "user count is ",count}' /etc/passwdCopy the code

  • BEGIN and END blocks
Awk -f ":" 'BEGIN{print "}{print "username: Username: root username: XXXXX username: root username: XXXXX username: root username: XXXXX username: root username: XXXXX username: root username: XXXXX username: XXX username: XXXXX This statement is executed after the processing is complete. It is usually used for statisticsCopy the code

2. Sort command

The sort command compares each line of a file as a unit, from the first character to the ASCII code value, and outputs them in ascending order.

1. Sort syntax format

Sort [- bcdfimMnr] [-o output file < >] t < space > characters [-] [+ < initial field > - < field > end] [-- help] [- verizon] [file]Copy the code

2. Common options

-f: Ignores case differences. For example, A and A are regarded as the same encoding. -b: Ignores the first space. -m: sorting by month name, such as JAN, DEC, etc. -n: use "numeric" sorting (default is literal). -r: in reverse order; -u: is uniq, the same data, only one row represents; -t: indicates the delimiter, which is separated by the [TAB] key by default. -k: sorted by that range (field)Copy the code

3, case

Apple,5 2.6 orange,8 8 banana,3 7.2 pear,6,4 [www@test /]$sort a.log apple,5 2.6 banana,3 7.2 orange,8,8 Pear,6,4 [www@test /]$sort -r a.log pear,6,4 orange,8,8 banana,3,7.2 apple,5,2.6 [www@test /]$sort -t, -k 2 a log # banana,3,7.2 apple,5,2.6 pear,6,4 orange,8,8Copy the code

Uniq command

The uniq command removes duplicate lines from sorted files, so it is often used with sort. That is, in order for UNIQ to work, all repeating rows must be contiguous.

1, UNIq syntax format

Uniq [- cdu] [-f < field >] [-s < character position >] [-w < character position >] [-- help] [-- version] [input file] [output file]Copy the code

2. Common parameters

-i: ignores the difference between upper and lower case characters. -c: performs counting. -u: displays only unique rowsCopy the code

3, case

B. log content banana,3,7.2 apple,5,2.6 pear,6,4 orange,8,9 apple,5,2.6 banana,3,7.2 [www@test /]$uniq-c b.log One banana, three,7.2 1 apple, 5,2.6 1 pear, 6, 4 1 orange, 1 apple 8 or 9, 5,2.6 1 banana, 3,7.2 / WWW @ test / $sort b.l og | uniq -c 2 apple,5 2.6 2 banana,3 7.2 1 orange,8 9 1 pear,6 4 www@test /]$Copy the code

4. Wc command

Used to count the number of words.

1. Wc syntax format

Wc [-clw][--help][--version][file...]Copy the code

2. Common parameters

-l: lists only the rows. -w: How many words (English words) are listed only; -m: How many characters;Copy the code

3, case

www@test /]$wc -l b.log # Count the number of lines in the b.log file 6 b.log www@test /]$Copy the code

5. Xargs command

Xargs is a powerful command that captures the output of one command and then passes it to another. Is to this command, the key is because many command does not support | pipe to pass parameters, and daily work have the necessary, so the xargs command

1. Format of the xargs command

somecommand |xargs -item  command
Copy the code

2. Common options

  • -a file Reads from the file as stDIN
  • -e flag, note that sometimes it may be -e. Flag must be a flag separated by Spaces. Xargs stops when it contains a flag.
  • -p Asks the user each time an argument is executed.
  • -n num specifies the number of arguments used at a time. By default, all arguments are used.
  • -t indicates that the command is printed first and then executed.
  • -i or -i, depending on Linux support, assigns each xargs item name, usually line by line, to {}, which can be replaced by {}.
  • -r no-run-if-empty Stops xargs when the input of xargs is empty.
  • -s num Indicates the maximum number of characters in the command line. It indicates the maximum number of characters in the command line following xargs.
  • -l num Reads the num line from the standard input once and sends the command to command.
  • -l and -l.
  • -d delim delimiter. The default xargs delimiter is carriage return. The default argument delimiter is space.
  • -x exit means, mainly used with -s.

3, case

# a. og content a b c d e f g h I j k l m n o p q r s t u v w x y z WWW @ $cat test /] a. og | xargs - n3 # - n option multi-line output a b c d e f g H I j k l m n o p q r s t u v w x y z $# WWW @ test /] echo "name# name# name# name" | xargs - d# # -d option can customize a delimiter name name Name the name WWW @ test /] $echo "name# name# name# name" | xargs d# - n2 name name name name # an option of xargs - I, Use -i to specify a replacement string {}, which is replaced when xargs is extended. When -i is used with xargs, each parameter command is executed once: WWW @ $cat test /] a. og | xargs -i echo {} {} apple, 5,2.6 orange, 8, 8 banana, 3,7.2 pear, 6, 4Copy the code

6. Grep command

Grep is a powerful text search tool that uses regular expressions to search for text and print the matching lines.

1. Format of the grep command

Grep [-acinv] [--color=auto] 'search string' filenameCopy the code

2. Common options

-c: count the number of times the 'search string' is found. -I: ignore the case difference, so the case is the same. -n: print the line number by the way. --color=auto: Displays where you can add color to the keywords you findCopy the code

3, case

#a.log Content XXXXXXXXXXXXXXXXXXXXXXX,user_id=999, XXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXX XXXXXXXXXXXX xxxxxxxxxxxxxxx,name=Lan,xxxxxxxxxxxxxxxxx xxxxxxxxxxx,user_id=999,xxxxx $ grep -n --color=auto "999" a.log 2:xxxxxxxxxxx,user_id=999,xxxxxxxxxxxxxxxx 6:xxxxxxxxxxx,user_id=999,xxxxx $ grep -c --color=auto "999" a.log 2 $ grep -i --color=auto "LAN" a.llog XXXXXXXXXXXXXXX,name= LAN, XXXXXXXXXXXXXXXXX $grep -i --color=auto "a.llogCopy the code

Welcome to pay attention to my subscription number, will regularly share some software testing related articles, questions are also welcome to discuss learning!

​

\

​