Why learn the Shell scripting language?

Almost all the underlying systems of modern Internet architecture are built based on Linux operating system. The core value of Linux lies in providing powerful system kernel functions for file management and information interaction management.

The Shell is a tool and bridge for software developers to effectively control and use Linux. Shell itself is system software written in C language, usually also called command line tool. It has a basic interface through which users access Linux operating system kernel services through Shell Script.

Tips: Although we often refer to “Shell” and “Shell scripting language” as “Shell,” there are fundamental differences between the two.

In science fiction movies, Shell scripts are often seen scrolling fast on dark screens, which has become a Geek/Hacker icon. In practice, Shell is also favored by development, operation and maintenance, testing and even operation personnel, and is almost a necessary skill for IT technicians. In the field of software testing technology, Shell script programming is the cornerstone of automated testing technology and is a necessary skill for test development engineers.

What are the advantages of Shell scripting?

The advantage of Shell scripting language is that it can process the low-level services of Linux operating system in the lightest and fastest speed. Such as automatic software installation, update version, monitoring alarm, log analysis and so on. Other high-level programming languages, such as PHP, Python, Ruby, etc., can also do this, but the efficiency and development cost are greatly reduced.

Mature technical people will abandon flashy methods, and will choose the most appropriate tools to solve problems according to different scenarios, simple and efficient. Grep, awk, and sed are the most efficient tools for Linux text processing problems.

Below, we will introduce the basic syntax, usage scenarios and features of the Three Musketeers of Linux text processing, as well as give the corresponding practice topics.

Shell programming environment installation

  • You are advised to install Git Bash for Windows users.

  • For Mac users, you are advised to install Iterm2.

  • SSH tools

  • Students at Hogwarts Will be able to log into their SSH mobile phone number using their personal account @shell.testing-studio.com

  • If you do not have an SSH account, you can temporarily use SSH [email protected]

  • The walk-through: testerhome.com/tmp/nginx.l… Save a one-day Nginx access log.

Linux three Musketeers introduction

  • Grep: used to search text content and support regular expressions.
  • Awk: mainly used for text analysis and processing, but also used to process data and generate reports. It is very suitable for data that needs to be processed by column. (A lot of Linux now uses GAWk)
  • Sed: full name Stream Editor, mainly used for editing text content. By default, only the schema space is processed, and the original data is not changed. Sed uses line by line reading to process data.

grep

Grep Example Shell script code

ps -ef  | grep bash
echo "ABC" | grep -i  abc
ps -ef | grep bash | grep -v grep
echo "1234 7654" | grep -o "[0-9]4"  
echo "1234 7654" | grep -oE "[0-9]4|76"
Copy the code

Grep Exercises

  • Find all the 404 and 503 error logs in nginx.log, take out the first 3 and post the command in the reply.

  • Find all HTTP and HTTPS links on the TesterHome home page.

awk

Awk sample Shell script code

ps | awk 'BEGIN{print "start"}{print $0}END{print "end"}' awk '/ 404 | 500 /' /tmp/nginx.log echo '1 2 3 4 5' | awk '/2/,/4/' echo '1 2 3 4 5' | awk '$0>3' ps | awk 'NR>1' ps | awk '{print $NF}' echo $PATH | awk 'BEGIN{RS=":"}{print $0}' | grep -v "^$" | awk 'BEGIN{FS="\n"; ORS = ":"} {print $0} END {printf "\ n"} 'echo' 1, 2, 3, 30 '| awk' BEGIN {a = 0; FS=","}{a+=$2}END{print a,a/NR}' awk 'BEGIN{print 33*20*76/200/3}' echo "123|456_789" | awk 'BEGIN{FS="\\||_"}{print $2} 'echo "_789" 123 | 456 | awk "BEGIN {FS = \" \ \ \ \ | | _ \ "} {print $2} \ "# use single quotation marksCopy the code

Awk combat exercises

  • Find the data for 404 and 500, print only the status code column, and sort to de-duplicate. Post the order in the reply
  • Go to the Home page of Testerhome and find all HTTP connections, then print the domain name only section without HTTP

sed

The pattern expression

  • 20, 30,35 lines and range of lines

  • /pattern/ Regular matches

  • //,// The interval of the regular match

action

  • D to delete

  • P printing, smooth combined with -n parameter

  • s/REGEXP/REPLACEMENT/[FLAGS]

  • Replace by referencing the fields that match \1 \2

Sed sample Shell script code

Ps | sed - n 1, 3 p ps | sed 's/CMD/command/' ps | sed - n'/ps/p 'echo' 1, 2, 3, 4, 5 '| sed - n' / 3 /, / / 4 p 'echo' 1, 2, 3, 4, 5 '| sed '/3/,/4/d' ps | sed -e 's/CMD/command/' -e 's#00#20#g'Copy the code

Sed

  • For all 404 500 data, the statistics of urls with such status code need to summarize the URL (summarize similar urls, remove the same resource but changed ID), print the top 5 problematic paths, and post the command in the reply.

  • (Article from Hogwarts Testing Institute)

    Click on the link for more articles

    Qrcode.testing-studio.com/f?from=juej…