find

Find is used to search for files

Commonly used search

-max-depth 2 Search directory depth not less than level 2 find. -min-depth 2 Search directory depth not less than level 2 find. -name '*.log' The file name matches, -iname "*.log "-iname ignore case find. -path '*m*/.log' path match, which is more fuzzy than the -name search. -ipath -ipath ignore case find. -size -10m Searches for directories or files whose size exceeds 10M. -10m searches for directories or files whose size exceeds 10M. -ctime +3 Searches for directories or files whose file properties have been changed more than 3 days, expressed in days. Modlfy time) find. -cmin +3 Queries directories or files whose file properties have been changed for more than 3 minutes, in minutes. -user root Queries the owning user, -group queries the owning group find. -perm 755 Queries files or directories with permission 755 find. -empty Queries all empty files in the directoryCopy the code

Regular match

Find. -name "[a-z].log" Matches a single character in the directory [] ending with. Log. -name "[a,b,d].log" Matches a single character in [] of the file ending with. Log. Find. -o is or -a is and! -empty-type f -exec rm -f {} \; Find. -empty-type f -ok rm -f {} \; Before every delete user interaction to confirm the find. - empty -type f | xargs rm -f query the current directory empty file and delete the find. / -type f - printf "% f \ n" obtain the relative path to the basename a. og b.log c.log d.log find . / -type f ! - path. /a.log Deletes a.log from the search resultsCopy the code

grep

Grep matches text or multiple regular expressions and outputs only lines or text that match (or do not match) for search purposes

The main parameters

-i Ignores case and case. -C Displays the number of matched lines. -n Displays the number and number of matched lines. -s Ignores errors that do not exist in the file. -r Recursively queries the matched characters in the directory. -L Displays only the file names that contain the matches. Matches with wc -l to count the number of times a word is matched. -p uses Perl regularCopy the code

Regular expression metacharacters

^ Anchor line header \< or \b anchor word header$Anchor the end of each lineMatches any character * matches zero or more preceding characters \? \{m,n\} matches the preceding characters at least m times, At most n times [] Matches a character within the specified range [^] Matches any character outside the specified range [[:digit:]] Matches any digit [[:lower:]] Matches any lowercase letter [[:upper:]] Matches any uppercase letter [[:alpha:]] Matches any single letter [[:alnum:]] Matches any single letter or number [[:punct:]] Matches any single symbol [[:space:]] Matches a single spaceCopy the code

sed

It is mainly used to automatically edit one or more files, simplify the repeated operation of files, and write conversion programs. Mainly used for editing

Parameters that

-i Directly modify the file content -n Mask the default output -r Enable extended regular expression. If used together with other options, it should be the first option. -{} Multiple commands can be combined, separated by,Copy the code

Processing action

P print line D delete line S replace string g global replace N Print the next line! If no, the value is reversed. = Print the line number. I Ignore case and match the string itselfCopy the code

Common operation examples

## # # # output linesSed -n '$p' sed -n '$p' sed -n '$p' sed -n '$p' sed -n '$p' sed -n '$p' sed -n '$p' sed -n '$p' sed -n '$p' 4p; Sed -n '/^root/p' TXT sed -n '/^root/p' TXT sed -n '/^root/p' TXT sed -n '/^root/p' /^m/p' TXT Outputs lines starting with root or m sed -n 'p; N 'TXT output odd lines,'n; Sed -n '$=' TXT ' Wc -l Output line number plus file name sed -n '/^device /,//p' sed -n '/ interval start identifier /,/ interval end identifier /p' oldboy.log sed 's/swap/No/Ig' ignore case replacement
## # # # delete rowsSed '/^$/d' TXT sed '/[0-9]/d' TXT sed '/nologin$/! D 'TXT to delete all the lines at the end of the nologin sed' / # / \ | ^ ^ $/ d 'TXT delete all comments and blank lines, \ | sed'/bin/d; /root/d 'txt Delete all lines containing 'bin','root'
## # # # to replaceSed 's/ XML/ XML/' TXT Replace the first XML in each line with XML sed 's/ XML/ XML/3' TXT Replace the third XML in each line with XML sed '1s/ XML/ XML/3' TXT Replace the third XML in the first line with XML Sed 's/doc/&s/g' TXT replace docs with docs Sed '1,4s/^#//' TXT 'sed '1,4s/^# root/' TXT' sed '1,4s/^# root/' TXT 'sed '1,4s/^# root/' TXT' sed '1,4s/^# root/' TXT 'sed '1,4s/^# root/' TXT 's/XML \ | XML \ | e / / g' TXT to delete all of XML, XML, sed string 'e' '/ ^ the HOSTNAME/cHOSTNAME: w' TXT will replace the HOSTNAME whole row for the HOSTNAME: w, Sed '/^HOSTNAME/aHOSTNAME'
##### comprehensive applicationSed 's/.//2; Sed -r 's/(.) sed -r 's/(.) sed -r 's/(.) (.). (. *) / \ \ \ 1 2 3 / 'TXT to delete all the data in the file, a blank line sed -r' s / [0-9] | ^ $/ / g 'TXTCopy the code

awk

Awk is a powerful analysis tool. Simply put, AWK reads each line one by one. By default, awK slices each line with a space delimiter

parameter

-f Specifies the delimiter for processing text -v setting variables. For example, -va=20, A can use -f in the command to specify awK script processing files, for example, awk -f a.awk passwdCopy the code

The operator

= assignment? Conditional expression: C | | logic or && logic and to match the regular expression < = = = > =! = Relational operators space join + - plus or minus * / % multiply, divide, remainder ^ ** exponentiate ++ -- increase or decrease in array membersCopy the code

Built-in variables

ARGC Number of command line parameters ARGV array of command line parameters ENVIRON System environment variables supported in the queue Apply FILENAME awk file getLine to obtain the next line FS set delimiter NR Total number of input files. FNR Total number of input files. OFS Specifies the field delimiter for output$0 indicates the entire record
$1 represents the first field of the current row, and so on
$NF represents the information in the last column
$NR is the first column of the first row, the second column of the second row, the third column of the third row. And so on
Copy the code

Common Commands

Awk '/root/' passwd Searches for all lines with root in the text awk -f: '/root/{print $7}' passwd Search for all root lines in the text bash awk 'NR==2 {print $0}' passwd print the second line awk -f: '$3>90' passwd prints the third row greater than 90 awk -f: Awk 'NR>13 && NR<17 {print $0}' awk -f [:/] '{print $1,$NF}' passwd Multiple delimiters are used without precedence. Nologin awk 'BEGIN{}{}END{}' start {}, END{} awk -f: '$1 ~ /^r/' {print $0}' matches the first row of the string starting with r awk 'BEGIN{IGNORECASE=1} /this/' IGNORECASE awk -f: 'the BEGIN {OFS = ":"} {print $3, $5}' specified according to the output of the separator awk - F: 'BEGIN{RS="\n\n"}{print $1}' specifies two newlines as line separators awk 'length>=70' passwd Prints lines larger than 70 awk '/^device /,//' Takes the interval value, device{} the contents of this module. /,/ indicates the content of the intervalCopy the code

Awk calculation

Awk -f: '{$3>= 900}END{print NR}' awk -f: '{$3>= 900}END{print NR}' 'BEGIN{a=0}{if ($3>a) a=$3}{print a}' passwd#Evaluate the maximum value in the column (assign a value greater than a to a)awk '{for (i=2; i<=NF; ++i) sum+=$i}{print $0,sum}{sum=0}'#Evaluate the row, sum=0, and return sum to 0 after each row

awk '{h[$1]+=$2}END{for(pol in h)printpol,h[pol]}' array_add.txt
#statisticalThe $1Following the same$2The numbers are added together and printed in alphabetical order

awk -F[/" ":] '{a[$4":"$5"\t"$8]++}END{for (ip in a) print a[ip],ip}' a|sort -k2|column -t
#Count the number of accesses per IP address per minute.$4.A $5As the time,$8For IP
Copy the code

Awk built-in functions

System executes a particular command and returns its exit status. If 0 is returned, the command is successfully executed. Non-0 indicates that the command fails to be executed# awk 'BEGIN { ret = system("date"); print "Return value = " ret }'Systime gets the timestamp and returns the number of whole seconds from January 1, 1970 to the current time (excluding leap years)# awk 'BEGIN{now=systime(); print now}'Strftime (fotmat,timestamp) Formats the timestamp# awk 'BEGIN {print strftime("Time = %m/%d/%Y %H:%M:%S", systime())}'
#M / % D % D equivalent to % % % % % F equivalent to y y - m - % D % % R is equivalent to the H: % m % % u said week j said the first day of the yearPrintf formatted string output, %d decimal signed integer % U decimal unsigned integer %f floating point %s String %-10s indicates left aligned 10 characters,$+20s indicates right aligned %p pointer value %g automatically select the appropriate format Gsub [old,new,[In]] gsub is a global replacement sub sub function performs a substring replacement. It replaces the first occurrence of the substring with a regex. The third argument is optional and defaults to $0 length The length of the string, which defaults to $0# awk -F: 'length($7)<10' passwdTolower converts all strings to lowercaseCopy the code

Awk conditions and loops

if

# ifThe statement formats
# if(Conditional expression)
#Action 1
# else if(Conditional expression)
#Action 2
# else:
#Movement 3The sample#Separated by:, only the information in the third column ranges from 50 to 100
awk 'BEGIN{FS=":"}{if($3>50 && $3<100) print $0}' passwd

Copy the code

for

for (initialisation; condition; increment/decrement) 
    action
# forThe statement first performs initialisation and then checks the condition.If the condition is true, an action is performed, and then a increment or decrement operation is performed to keep the cycle going as long as the condition is true. At the end of each loop, a condition check is performed, and the loop is terminated if the condition is false. Example awk 'BEGIN {for (I = 1; i <= 5; ++ I) print I}Copy the code

while

The syntax of the While While loop is as follows: While (condition) Action The While loop first checks if the condition is true, and if it is true then the action is executed. This process is repeated until the condition is flase awk 'BEGIN{I =1; while (i<5) {print i; I++}}' 1 2 3 4Copy the code

break

14. Break is used to end a loop:$ awk 'BEGIN {
   sum = 0; for (i = 0; i < 20; ++i) { 
      sum += i; if (sum > 50) break; else print "Sum =", sum 
   } 
}'
Copy the code

continue

The Continue statement is used to end the loop inside the body of the loop, leading directly to the next iteration of the loop.$ awk 'BEGIN {for (i = 1; i <= 20; ++i) {if (i % 2 == 0) print i ; else continue} }'
Copy the code

exit

Exit Exits the loopCopy the code

paste

Paste is used to combine two files

The main parameters

-s changes the file to a line. -d specifies a newline characterCopy the code