Shell scripts are a powerful tool to help programmers and system administrators accomplish tedious tasks that take time and effort. They are an effective way to interact with computers and manage files and system operations. Just a few lines of code can get the computer close to doing what you want.

The migrant worker brother sorted out 100 examples for everyone, through 100 actual combat classic script examples, demonstrated the shell script programming practical technology and common tool usage. All you need to do is apply these common tasks and portable automation scripts to other similar problems that you might encounter every once in a while, depending on your needs.

Checks for file consistency in the directory specified by the two servers

#! / bin/bash # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # to detect two servers specified directory file consistency # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # Dir =/data/web b_ip=192.168.88.10 And written to the specified file find $dir -type f | xargs md5sum > / TMP/md5_a. TXT SSH $b_ip "find $dir -type f | xargs md5sum > / TMP/md5_b. TXT" For f in 'awk '{print 2} / TMP /md5_a.txt'' do # for f in 'awk '{print 2} / TMP /md5_a.txt'' do When b machine directly when there is no traverse object files in the output does not exist if the result of the grep -'ve "$f"/TMP/md5_b. TXT then md5_a = ` grep -w "$f"/TMP/md5_a. TXT | awk '{1} print ` Md5_b = ` grep -w "$f"/TMP/md5_b. TXT | awk '{1} print ` # when file exists, If [$md5_a!= $md5_b] then echo "$f changed." $f else echo "$f deleted.

Regular empty file content, regular record file size

#! / bin/bash # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # execute a script (task plan) per hour, when the time for zero or twelve o 'clock, Empty all files in the target directory, but do not delete files. At other times, only count the size of each file, one line for each file, and output it to the file named by the time and date. Need to consider the target directory under the secondary and tertiary DengZi directory file # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # logfile = / TMP / ` H - date + % % F `. The log N = ` date + % H ` if [$n - eq 00] | | [$n - eq 12] then # by the for loop, as a condition of traverse to find command, For I in 'find /data/log/ -type f' do true bb0 $I done else for I in 'find /data/log/ -type f' do du  -sh $i >> $logfile done fi

Detect network card traffic and record it in the log according to the specified format

#! / bin/bash # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # detection of network traffic, and recorded in the log as a specified format # # once a minutes record log format prescribed as follows: #2019-08-12 20:40 #ens33 input: 1234bps #ens33 output: 1235bps ######################################################3 while : Do # set the language to English, ensure that the output is in English, LANG=en logfile=/ TMP/' date +%d '. Log # Redirect output to logfile log exec >> $logfile date +"%F %H:%M" The # SAR command counts traffic in KB /s and logs in BPS format. So to 59 * 1000 * 8 SAR - n DEV 1 | grep business | grep ens33 | awk '{print $2,"\t","input:","\t",$5*1000*8,"bps","\n",$2,"\t","output:","\t",$6*1000*8,"bps"}' echo "####################" # Because it takes 59 seconds to execute the SAR command, no sleep done is required

Count the number of numbers that appear in each line of the document, and count the total number of numbers in the entire document

#! / bin/bash # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # to calculate the number of digits in a document appear in a row, And calculate the total number of digital document # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # use awk only output lines (interception of the first paragraph) n = ` wc -l a.t xt | awk '{print For I in 'seq 1 $n' do # for I in 'seq 1 $n' do # for I in 'seq 1 $n' do # for I in 'seq 1 $n' do # for I in 'seq 1 $n' do # for I in 'seq 1 $n' do # for I in 'seq 1 $n' do # for I in 'seq 1 $n' do # for I in 'seq 1 $n' do # Statistics the longest length n_n = ` echo $line | sed s' / [^ 0-9] / / 'g | wc -l ` echo $n_n sum = $[$sum + $n_n] done echo "sum: $sum"

Kill all scripts

#! / bin/bash # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # has some scripts to join into the cron, existing script has not yet run out and new tasks need to be performed, # causes the system to load up, so you can write scripts to screen out processes that affect the load and kill them all at once. # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # ps aux | grep specify the process name | grep -v grep | awk '{print $2}' | xargs kill -9

Download the file from the FTP server

#! /bin/bash if [ $# -ne 1 ]; then echo "Usage: $0 filename" fi dir=$(dirname $1) file=$(basename $1) ftp-n-v << EOF # -n open 192.168.1.10 # FTP server user admin CD $dir get "$file" EOF CD $dir get "$file" EOF

Enter five consecutive numbers less than 100, and count the sum, minimum, and maximum

#! /bin/bash COUNT=1 SUM=0 MIN=0 MAX=100 while [ $COUNT -le 5 ]; Do read -p "please enter 1-10 integers:" INT the if [[! $INT = ~ ^ [0-9] + $]]. Then echo "Input must be integer!" exit 1 elif [[ $INT -gt 100 ]]; Then echo "Input must be less than 100!" exit 1 fi SUM=$(($SUM+$INT)) [ $MIN -lt $INT ] && MIN=$INT [ $MAX -gt $INT ] && MAX=$INT let COUNT++ done echo "SUM: $SUM" echo "MIN: $MIN" echo "MAX: $MAX"

Users guess the number

#! The /bin/bash # script generates a random number within 100 and prompts the user to guess the number. Based on the user's input, the script prompts the user to guess the number correctly. The script prompts the user to guess the number correctly. ‐32767 = 1‐100 ‐ num=$[RANDOM%100+1]echo "$num" # use read to prompt the user to guess the number # use if ‐eq(equal to),‐ne(not equal to),‐gt(greater than),‐ge(greater than or equal to), # ‐lt(less than),‐le(less than or equal to) while: do read-p "the computer generated a random number of 1‐100, you guessed it: "CAI if [$CAI - > eq $num] then echo" exit elif [$CAI - > $num] then echo "else echo done

Monitor NGINX access log 502 and take action accordingly

Assuming the server environment is LNMP, 502 phenomenon often occurs in recent visits, and 502 error disappears after restarting PHP-FPM service, so it is necessary to write a monitoring script. Once 502 occurs, PHP-FPM service will be automatically restarted.

The path to the log file is /data/log/access.log #2. The script loops endlessly, detecting once every 10 seconds. The number of log entries in 10 seconds is 300. If the percentage of 502 entries is at least 10% (30 entries), the PHP-FPM service #3 needs to be restarted. /etc/init.d/php-fpm restart #! / bin/bash # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # 502 monitoring Nginx access log book, And do the corresponding action # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # log = / data/log/access. To set threshold value while the log N = 30 # : Do # View the latest 300 access logs, And statistics of 502 the number of times the err = ` tail - $300, log n | grep -c '502 "' ` if ge $n] [$err - then/etc/init. D/PHP - FPM restart 2 > / dev/null # set 60s delay to prevent script bug from causing unlimited restart of php-fpm service sleep 60 fi sleep 10 done

Assign the results to variables separately

Application scenario: You want to assign execution results or positional parameters to variables for later use.

Method 1:

for i in $(echo "4 5 6"); do
   eval a$i=$i
done
echo $a4 $a5 $a6

Method 2: Split the position parameter 192.168.1.1{1,2} into each variable

num=0 for i in $(eval echo $*); Let num+=1 eval node${num}="$I "done echo $node1 $node2 $node3 # bash a.sh192.168.1.1 {1,2} 192.168.1.11 192.168.1.12

Method 3:

arr=(4 5 6)
INDEX1=$(echo ${arr[0]})
INDEX2=$(echo ${arr[1]})
INDEX3=$(echo ${arr[2]})

Batch change file name

Example: # touch article_{1.. 3}. HTML # ls article_1. HTML article_2. HTML article_3. HTML purpose: changing the article to BBS method 1: for file in $(ls * HTML); do mv $file bbs_${file#*_} # mv $file $(echo $file |sed -r 's/.*(_.*)/bbs\1/') # mv $file $(echo $file |echo bbs_$(cut -d_ -f2) done method 2: for file in $(find.-maxdepth 1-name "* HTML "); Do mv $file bbs_${file#*_} done: # rename article BBS *.html

Delete the first five lines of a document that contain letters, and delete all letters on lines 6 to 10

1) Prepare the test file named 2.txt

Line 1 1234567 does not contain the letters line 2 56789BBBBBB line 3 67890CCCCCCCC line 4 78asdfddddddddddd line 5 123456eeeeee line 6 1234567ASDF line 7 56789ASDF Line 8 67890ASDF Line 9 78ASDFADSF Line 10 123456AAAA Line 11 67890ASDF Line 12 78ASDFADSF Line 13 123456AAAA

2) The script is as follows:

#! / bin/bash # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # to a document contained in the letter of the five elements to delete, At the same time deleted from six to 10 lines contain all the letters # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # sed - n '1, 5 p 2. TXT | sed' / [a zA - Z] / 'd Sed - n '6, 10 p 2. TXT | sed s' / [a zA - Z] / /' g sed - n '$' 11, p 2. TXT # end result is printed on the screen as a result, if you want to change the file directly, the output can be written to a temporary file, and then replace 2. TXT or using the -i option

Total size of files ending in.html in the current directory

# find.-name "*.html" -exec du -k {} \; | awk 'END {sum + = $1} {print the sum}' method 2: for the size in $(ls - l *. HTML | awk '{print $5}); do sum=$(($sum+$size)) done echo $sum

Scan host port status

#! /bin/bash HOST=$1 PORT="22 25 80 8080" for PORT in $PORT; do if echo &>/dev/null > /dev/tcp/$HOST/$PORT; then echo "$PORT open" else echo "$PORT close" fi done

Use the shell to print words with fewer than 6 letters in the example statement

#Bash also interprets a number of multi-character options. #! / bin/bash # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # shell print sample statement letter number less than 6 words ############################################################## for s in Bash also interprets a number of multi-character  options. do n=`echo $s|wc -c` if [ $n -lt 6 ] then echo $s fi done

Enter the number to run the corresponding command

#! / bin/bash # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # input digital operation commands accordingly ############################################################## echo "*cmd menu* 1-date 2-ls 3-who 4-pwd 0-exit " while : The do # capture user type value read -p "both please input the number:" n n1 = ` echo $n | sed s' / [0-9] / / g ` # empty input check if [-z "$n] then continue fi If [-n "$n1"] then exit 0 fi break done case $n in 1) date; 2) ls ;; 3) who ;; 4) pwd ;; 0) break ;; Echo "please input number is [1-4]" esac

Expect implements SSH interactive-free command execution

Expect is a tool for automating interactive applications such as Telnet, FTP, passwd, and so on. You need to install the Expect package first.

Method 1: EOF standard output as Expect standard input #! /bin/bash USER=root PASS=123.com IP=192.168.1.120 expect << EOF set timeout 30 spawn SSH $USER@$IP expect {"(yes/no)" {send "yes\r"; exp_continue} "password:" {send "$PASS\r"} } expect "$USER@*" {send "$1\r"} expect "$USER@*" {send "exit\r"} expect eof EOF method 2: #! /bin/bash USER=root PASS=123.com IP=192.168.1.120 expect -c "spawn SSH $USER@$IP expect {\"(yes/no)\" {send \"yes\r\";  exp_continue} \"password:\" {send \"$PASS\r\"; exp_continue} \"$USER@*\" {send \"df -h\r exit\r\"; Exp_continue}}" Method 3: Separate Expect script /usr/bin/expect set ip [lindex $argv 0] set user [lindex $argv 1] set passwd [lindex $argv 2] set cmd [lindex $argv 3] if { $argc ! = 4 } { puts "Usage: expect login.exp ip user passwd" exit 1 } set timeout 30 spawn ssh $user@$ip expect { "(yes/no)" {send "yes\r"; exp_continue} "password:" {send "$passwd\r"} } expect "$user@*" {send "$cmd\r"} expect "$user@*" {send "exit\r"} expect EOF execute command script: write a loop can batch operation of multiple servers #! /bin/bash HOST_INFO=user_info.txt for ip in $(awk '{print $1}' $HOST_INFO) do user=$(awk -v I="$ip" 'I==$1{print $2}' $HOST_INFO) pass=$(awk-v I="$IP "'I==$1{print $3}' $HOST_INFO) expect login.exp $IP $user $pass $1 done # cat user_info.txt 192.168.1.120 root 123456

Create 10 users, and set the password respectively, password requires 10 characters, including upper and lower case letters and numbers, finally need to store each user’s password in the specified file

#! / bin/bash # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # to create 10 users, and set the password, Password should be 10 characters, including upper and lower case letters and numbers # Finally, you need to store each user's password in the specified file # Prerequisite: Install mkpasswd command # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # to generate the sequence of 10 users (00-09) for u in ` seq -w 0 09 ` do # # to create user useradd user_ $u generated password p = ` mkpasswd -s 0 to 10 ` # l read from standard input password modification (unsafe) echo $p | passwd -- stdin user_ $u # regular change passwords echo - e "$p $p" \ n | passwd user_ $# u will create the user and the corresponding password record to the log file echo "user_ $u $p > > / TMP/userpassword done

Monitor the number of processes in HTTPD and act accordingly

#! /bin/bash # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # ####### # Requirements: #1. If the number of processes is greater than or equal to 500, then automatically restart the Apache service and check whether the service restarted successfully #2. If it is not successful, it needs to be restarted. If it is still not successful after 5 restarts, it will send an alert email to the administrator and exit test #3. If the startup is successful, wait for 1 minute to check the number of HTTPD processes again. If the number of processes is normal, the normal detection will be resumed (once every 10s), otherwise the restart will be abandoned and an alert email will be sent to the administrator. And exit the test # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # check_service counter function () {j = 0 for I ` in seq # 1 5 ` do restart Apache command/usr/local/apache2 / bin/apachectl restart 2 > If [$j+1] then break else =$[$j+1] if [$j+1] then break else =$[$j+1] then mail.py exit fi done } while : Do n = ` pgrep -l HTTPD | wc -l ` # whether service HTTPD process for more than 500 if [$n - gt 500] then/usr/local/apache2 / bin/apachectl restart if [$? - ne 0] then check_service else sleep (n = 60 ` pgrep -l HTTPD | wc -l ` # whether after the restart still more than 500 if [$n2 - gt 500] then mail. Py exit Fi fi fi # detect every 10s sleep 10 done

Batch modifying server user passwords

Linux host SSH connection information: Old password

# cat old_pass.txt 192.168.18.217 root 123456 22 192.168.18.218 root 123456 22

Remote password change script for SSH: New password is generated randomly

#! /bin/bash OLD_INFO=old_pass.txt NEW_INFO=new_pass.txt for IP in $(awk '/^[^#]/{print $1}' $OLD_INFO); do USER=$(awk -v I=$IP 'I==$1{print $2}' $OLD_INFO) PASS=$(awk -v I=$IP 'I==$1{print $3}' $OLD_INFO) PORT=$(awk -v I=$IP Echo "$IP $USER $NEW_PASS $PORT" >> $NEW_INFO expect -c"  spawn ssh -p$PORT $USER@$IP set timeout 2 expect { \"(yes/no)\" {send \"yes\r\"; exp_continue} \"password:\" {send \"$PASS\r\"; exp_continue} \"$USER@*\" {send \"echo \'$NEW_PASS\' |passwd --stdin $USER\r exit\r\"; # cat new_pass.txt 192.168.18.217 root n8wX3mU% 22 192.168.18.218 root c87; ZnnL 22

Iptables automatically blocks IP addresses that visit your site frequently

Scenario: Malicious access, security precautions

  • 1) Block more than 200 IP visits per minute

Approach 1: Based on the access log (Nginx for example)

#! /bin/bash DATE=$(date +%d/%b/%Y:%H:%M) ABNORMAL_IP=$(tail -n5000 access.log |grep $DATE |awk '{a[$1]++}END{for(i in A)if(a[I]>100)print I}') # tail AWK cannot filter the log directly because it contains special characters. for IP in $ABNORMAL_IP; do if [ $(iptables -vnL |grep -c "$IP") -eq 0 ]; then iptables -I INPUT -s $IP -j DROP fi done

Approach 2: Connections established over TCP

#! /bin/bash ABNORMAL_IP=$(netstat -an |awk '$4~/:80$/ && $6~/ESTABLISHED/{gsub(/:[0-9]+/,"",$5); {a[$5]++}}END{for(I in a)if(a[I]>100)print I}') # print I}'; do if [ $(iptables -vnL |grep -c "$IP") -eq 0 ]; then iptables -I INPUT -s $IP -j DROP fi done
  • 2) Block IP that tries to log in more than 10 times per minute

Method 1: Get login status from lastb:

#! /bin/bash DATE=$(DATE +"%a %b %e %H:%M") # $(DATE +"%a %b %e %H:%M") And % d display 07 ABNORMAL_IP = $(lastb | grep "$DATE" | awk '+ +} {a ($3) END {for (I a) in the if (a [I] > 10) print I}') for IP in $ABNORMAL_IP; do if [ $(iptables -vnL |grep -c "$IP") -eq 0 ]; then iptables -I INPUT -s $IP -j DROP fi done

Method 2: Obtain the login status through the log

#! /bin/bash DATE=$(date +"%b %d %H! [](/img/bVbLc6v)") ABNORMAL_IP="$(tail -n10000 /var/log/auth.log |grep "$DATE" |awk '/Failed/{a[$(NF-3)]++}END{for(i in a)if(a[i]>5)print i}')" for IP in $ABNORMAL_IP; do if [ $(iptables -vnL |grep -c "$IP") -eq 0 ]; then iptables -A INPUT -s $IP -j DROP echo "$(date +"%F %T") - iptables -A INPUT -s $IP -j DROP" >>~/ssh-login-limit.log  fi done

Due to space constraints, there are only 22 of these scripts listed here, but I have compiled all the 100 scripts that are really useful. See the pictures

100 very useful shells to use script instances. PDF migrant worker brother has given you liver finished, now free to share with you, just in the migrant worker geo technology road public number backstage reply keywords [100 shell scripts] free access to script download address.