Play with the Linux operating system

Note: The explanation of Linux commands in this article is based on the Linux distribution named CentOS. I use Ali Cloud server, and the system version is CentOS Linux Release 7.6.1810. There are some differences between Linux distributions in Shell commands and utilities, but these differences are minor.

History of operating systems

Only hardware without software computer system is called “bare machine”, we can hardly use “bare machine” to complete the daily work of the computer (such as storage and operation), so must use specific software to control the work of the hardware. The software closest to computer hardware is system software, of which the most important is “operating system”. “Operating system” is a collection of programs that control and manage the entire computer hardware and software resources, realize resource allocation and task allocation, and provide interfaces and environments for system users and other software.

No operating system (manual operation)

In the early days of computers without operating systems, people put program tape (or card) on the computer, then start the input machine to put the program into the computer, and then through the console switch to start the program to run. When the program is finished, the printer outputs the results of the calculation, and the user removes and removes the tape (or card). The second user gets on the computer and repeats the same process. During the whole process, the user monopolizes the machine, and the CPU waits for manual operation, resulting in low resource utilization.

Batch system

First start a monitor program on the computer, under the control of the monitor program, the computer can automatically, batch processing of one or more users’ jobs. After completing a batch of jobs, the level of supervision reads the jobs from the input machine and saves them to the tape drive. Repeat the steps above. The supervisor process each job continuously, realizes the job automatic transfer, reduces the job establishment time and manual operation time, improves the utilization rate of computer resources. Batch system can be divided into single channel batch system, multi-channel batch system, online batch system, offline batch system.

Time-sharing and real-time systems

The time-sharing system divides the running time of the processor into very short time slices by which the processor is assigned to each online job in turn. If a job is unable to complete its calculations within the time slice allocated to it, the job is temporarily interrupted, leaving the processor to another job and waiting for the next round of scheduling to resume. Because computers are so fast, jobs rotate so quickly that each user feels like he has a computer to himself. And each user can issue various operation control commands to the system through their terminal, and complete the operation under the condition of full human-computer interaction. In order to solve the problem that the time-sharing system cannot respond to the user’s instructions in time, the real-time system that can complete the event processing in a strict time range and respond to random external events in time appears.

Universal operating system

1960S: IBM’s System/360 series of machines have a unified operating System, OS/360.
1965: AT&T’s Bell LABS joins GE and MIT in a joint program to develop MULTICS.
1969: When the MULTICS project fails and Ken Tompson is out of work, he develops Unics in assembly language on the then-obsolete PDP-7 to play the game “Space Travel”.

Note: It’s hard to imagine a system as great as Unix being developed by a stay-at-home programmer (with his wife and kids, to boot) on an obsolete device to play games on.
1970 ~1971: Ken Tompson and Dennis Ritchie rewrite Unics on PDP-11 in B language and rename it Unix at Brian Kernighan’s suggestion.

1972-1973: Dennis Ritchie invits C to replace the less portable B language and begins work on rewriting Unix in C.
1974: Unix releases its landmark fifth edition, implemented almost entirely in C.
1979: Beginning with Unix version 7, AT&T issues new terms of use, privatizing Unix.
1987: Professor Andrew S. Tanenbaum, in order to teach his students the ins and out of the way of operating systems, decides to develop a UniX-compatible operating system without using any AT&T source code to avoid copyright disputes. The system is called Minix.

In 1991: While studying at the University of Helsinki in Finland, Linus Torvalds tried to do some development on Minix, but because Minix was intended as an educational operating system, it was not very powerful. To read, write, and download files in the school’s newsgroups and mail system, Linus wrote disk drivers and file systems. These things formed the prototype of the Linux kernel.

Below is a map of the Unix operating system family.

Linux overview

Linux is a universal operating system. An operating system is responsible for task scheduling, memory allocation, handling peripheral I/O, and so on. An operating system usually consists of a kernel (the core program that runs other programs and manages hardware devices like disks and printers) and system programs (device drivers, underlying libraries, shells, service programs, and so on).

The Linux kernel was developed by Linus Torvalds of Finland and released in September 1991. As a product of the Internet era, Linux operating system is a free operating system developed by many developers around the world. (Note that free and free are not the same concept. If you want to know the difference, you can click here.)

Advantages of Linux

A general-purpose operating system that is not tied to specific hardware.
Written in C language, strong portability, a kernel programming interface.
Support multi-user and multi-task, support secure hierarchical file system.
Lots of utilities, complete networking features and strong support documentation.
Reliable security and good stability, more friendly to developers.

Linux distribution

Redhat
Ubuntu
CentOS
Fedora
Debian
openSUSE

Basic commands

Linux commands usually have the following format:

Command name [Named parameter] [Command object]Copy the code

Get the login information -w/who/last/ lastb.

[root ~]# w 23:31:16 up 12:16, 2 users, load average: 0.00, 0.01, 0.05 USER TTY FROM login@idle JCPU PCPU WHAT root PTS /0 182.139.66.250 23:03 4.00s 0.02s 0.00s w Jackfrue PTS /1 182.139.66.250 23:263:56 0.00s 0.00 s-bash [root ~]# who root PTS /0 2018-04-12 23:03 (182.139.66.250) jackfrued PTS /1 2018-04-12 23:26 (182.139.66.250) [root ~]# who am I root PTS /0 [root ~]# who mom Likes root PTS /0 2018-04-12 23:03 (182.139.66.250) [root ~]# last root PTS /0 117.136.63.184 Sun May 26 18:57 still Logged in Reboot system boot 3.10.0-957.10.1. Mon May 27 02:52-19:10 (-7:-42) Root PTS /4 117.136.63.184 Sun May 26 18:51-crash (08:01) root PTS /4 117.136.63.184 Sun May 26 18:49-18:49 (00:00) Root PTS /3 117.136.63.183 Sun May 26 18:35 - Crash (08:17) root PTS /2 117.136.63.183 Sun May 26 18:34 - Crash (08:17) root PTS /0 117.136.63.183 Sun May 26 18:10 - crash (08:42)Copy the code

Take a look at your shell-ps.

Shell is also known as “Shell” or “Shell program”. It is the translator for users to communicate with the operating system kernel. Simply speaking, it is the interface and interface for people to interact with computers. At present, the default Shell of many Linux systems is Bash (Bourne Again Shell), because it can use the TAB key to complete commands and paths, save historical commands, easily configure environment variables, and perform batch operations.
```
[root@izwz97tbgo9lkabnat2lo8z ~]# ps
  PID TTY          TIME CMD
 3531 pts/0    00:00:00 bash
 3553 pts/0    00:00:00 ps
Copy the code
```

View the command’s description and location – whatis/which/whereis.

[root ~]# whatis ps
ps (1)        - report a snapshot of the current processes.
[root ~]# whatis python
python (1)    - an interpreted, interactive, object-oriented programming language
[root ~]# whereis ps
ps: /usr/bin/ps /usr/share/man/man1/ps.1.gz
[root ~]# whereis python
python: /usr/bin/python /usr/bin/python2.7 /usr/lib/python2.7 /usr/lib64/python2.7 /etc/python /usr/include/python2.7 /usr/share/man/man1/python.1.gz
[root ~]# which ps
/usr/bin/ps
[root ~]# which python
/usr/bin/python
Copy the code

Clear what is displayed on the screen – clear.

Check the help documentation – man/info/help/apropos.

[root@izwz97tbgo9lkabnat2lo8z ~]# ps --help
Usage:
 ps [options]
 Try 'ps --help <simple|list|output|threads|misc|all>'
  or 'ps --help <s|l|o|t|m|a>'
 for additional help text.
For more details see ps(1).
[root@izwz97tbgo9lkabnat2lo8z ~]# man ps
PS(1)                                User Commands                                PS(1)
NAME
       ps - report a snapshot of the current processes.
SYNOPSIS
       ps [options]
DESCRIPTION
...
Copy the code

View the system and hostname – uname/hostname.
```
[root@izwz97tbgo9lkabnat2lo8z ~]# uname
Linux
[root@izwz97tbgo9lkabnat2lo8z ~]# hostname
izwz97tbgo9lkabnat2lo8z
[root@iZwz97tbgo9lkabnat2lo8Z ~]# cat /etc/centos-release
CentOS Linux release 7.6.1810 (Core)
Copy the code
```
Note: CAT is the command to connect the contents of a file and print them to standard output, as described later; /etc is a very important directory on Linux systems. It holds a lot of configuration files. Centos-release is a file in this directory, and since I use centos 7.6 for my own Linux distribution, I have one here.

Time and date – date/CAL.

[root@iZwz97tbgo9lkabnat2lo8Z ~]# date
Wed Jun 20 12:53:19 CST 2018
[root@iZwz97tbgo9lkabnat2lo8Z ~]# cal
      June 2018
Su Mo Tu We Th Fr Sa
                1  2
 3  4  5  6  7  8  9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
[root@iZwz97tbgo9lkabnat2lo8Z ~]# cal 5 2017
      May 2017
Su Mo Tu We Th Fr Sa
    1  2  3  4  5  6
 7  8  9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30 31
Copy the code

Restart and shutdown – reboot/shutdown.

[root ~]# shutdown -h +5
Shutdown scheduled for Sun 2019-05-26 19:34:27 CST, use 'shutdown -c' to cancel.
[root ~]# 
Broadcast message from root (Sun 2019-05-26 19:29:27 CST):

The system is going down for power-off at Sun 2019-05-26 19:34:27 CST!
[root ~]# shutdown -c

Broadcast message from root (Sun 2019-05-26 19:30:22 CST):

The system shutdown has been cancelled at Sun 2019-05-26 19:31:22 CST!
[root ~]# shutdown -r 23:58
Shutdown scheduled for Sun 2019-05-26 23:58:00 CST, use 'shutdown -c' to cancel.
[root ~]# shutdown -c

Broadcast message from root (Sun 2019-05-26 19:31:06 CST):

The system shutdown has been cancelled at Sun 2019-05-26 19:32:06 CST!
Copy the code

Note: when the shutdown command is executed, a warning message is sent to the user who logs in to the system. You can replace the default warning message with a warning message after the command, or use now after the -h parameter to indicate that the system is shutdown immediately.

Logout – exit/logout.
View the history command -history.

[root@iZwz97tbgo9lkabnat2lo8Z ~]# history
...
452  ls
453  cd Python-3.6.5/
454  clear
455  history
[root@iZwz97tbgo9lkabnat2lo8Z ~]# !454
Copy the code

After viewing the history command, you can use! History the command number to re-execute the command; You can clear the history command by running the history -c command.

utility

File and folder operations

Create/delete an empty directory -mkdir/rmdir.

[root ~]# mkdir abc
[root ~]# mkdir -p xyz/abc
[root ~]# rmdir abc
Copy the code

Create/delete files – touch/rm.
```
TXT [root ~]# touch error. TXT rm: remove regular empty file 'error. TXT'? y [root ~]# rm -rf xyzCopy the code
```
- touchThe command is used to create a blank file or change the file time. On Linux, there are three times for a file:
  - Change the time of the content – mtime.
  - Change permission time – ctime.
  - Last access time – atime.
- rmSeveral important parameters of:
  - -i: Interactive delete, each delete item will be queried.
  - -r: Deletes a directory and recursively deletes files and directories in the directory.
  - -f: Deletes files forcibly, ignoring non-existent files without prompting.
Switch and view the current working directory -cd/PWD.

Note: the CD command can be followed by a relative path (with the current path as a reference) or an absolute path (starting with a /) to switch to the specified directory, or you can use CD.. To return to the previous directory. What arguments should be added to the CD command if you want to go back to the previous directory?
View the directory -ls.
- -l: View files and directories in long format.
- -a: Displays files and directories starting with a dot (hide files).
- -RRecursively expand when encountering a directory (continue listing files and directories below the directory).
- -d: Lists only the contents, not other contents.
- -S / -t: Sort by size/time.

View the file content. -cat/tac/head/tail/more/less/rev/OD.

[root ~]# wget http://www.sohu.com/ -O sohu.html --2018-06-20 18:42:34-- http://www.sohu.com/ Resolving www.sohu.com (www.sohu.com)... 14.18.240.6 Connecting to www.sohu.com (www.sohu.com) | 14.18.240.6 | : 80... connected. HTTP request sent, awaiting response... [text/ HTML] Saving to: 'sohu.html'100%[= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = >] 212527 -. - K/sin0.03 s2018-06-20 18:42:34 (7.48 MB/s) - 'sohu.html' saved [212527/212527] [root ~]# cat sohu.html... [root ~]# head -10 sohu.html <! DOCTYPE HTML > < HTML > <head> <title> <meta name="Keywords" Content =" Sohu, Portal, New Media, Online media, News, Finance, Sports, Entertainment, Fashion, Cars, real estate, Technology, Photos, Forums, weibo, blogs, videos, movies, TV series "/> <meta name="Description" Content =" Sohu provides users with the latest news 24 hours a day, as well as search, email and other web services. The content includes global hot events, breaking news, current affairs comments, popular movies and TV series, sports events, industry trends, life service information, as well as forums, blogs, microblogs, my Sohu and other interactive Spaces. /> <meta name="shenma-site-verification" content="1237e4d02a3d8d73e96cbd97b699e9c3_1504254750"> <meta charset="utf-8"/> <meta http-equiv="X-UA-Compatible" content="IE=Edge,chrome=1"/> [root ~]# tail -2 sohu.html </body> </html> [root ~]# less sohu.html ... [root ~]# cat -n sohu.html | more ...Copy the code

Note: a command named wget is used above, which is a network downloader program that can download resources from the specified URL.

Copy/move the file -cp/mv.

[root ~]# mkdir backup
[root ~]# cp sohu.html backup/
[root ~]# cd backup
[root backup]# ls
sohu.html
[root backup]# mv sohu.html sohu_index.html
[root backup]# ls
sohu_index.html
Copy the code

Rename a file – rename.

[root@iZwz97tbgo9lkabnat2lo8Z ~]# rename .htm .html *.htm
Copy the code

Find files and search contents – find/grep.

[root@iZwz97tbgo9lkabnat2lo8Z ~]# find / -name "*.html" /root/sohu.html /root/backup/sohu_index.html [root@izwz97tbgo9lkabnat2lo8z ~]# find . -atime 7 -type f -print [root@izwz97tbgo9lkabnat2lo8z ~]# find . -type f -size +2k [root@izwz97tbgo9lkabnat2lo8z ~]# find . -type f -name "*.swp" -delete [root@iZwz97tbgo9lkabnat2lo8Z ~]# grep "<script>" sohu.html -n 20:<script> [root@iZwz97tbgo9lkabnat2lo8Z ~]# grep -E \<\/? script.*\> sohu.html -n 20:<script> 22:</script> 24:<script src="//statics.itc.cn/web/v3/static/js/es5-shim-08e41cfc3e.min.js"></script> 25:<script src="//statics.itc.cn/web/v3/static/js/es5-sham-1d5fa1124b.min.js"></script> 26:<script src="//statics.itc.cn/web/v3/static/js/html5shiv-21fc8c2ba6.js"></script> 29:<script type="text/javascript"> 52:</script> ...Copy the code

Note: Grep can be used to search a string using a regular expression. To use a regular expression, run grep -e or egrep.

Create links and view links -ln/readlink.
```
[root@iZwz97tbgo9lkabnat2lo8Z ~]# ls -l sohu.html
-rw-r--r-- 1 root root 212131 Jun 20 19:15 sohu.html
[root@iZwz97tbgo9lkabnat2lo8Z ~]# ln /root/sohu.html /root/backup/sohu_backup
[root@iZwz97tbgo9lkabnat2lo8Z ~]# ls -l sohu.html
-rw-r--r-- 2 root root 212131 Jun 20 19:15 sohu.html
[root@iZwz97tbgo9lkabnat2lo8Z ~]# ln /root/sohu.html /root/backup/sohu_backup2
[root@iZwz97tbgo9lkabnat2lo8Z ~]# ls -l sohu.html
-rw-r--r-- 3 root root 212131 Jun 20 19:15 sohu.html
[root@iZwz97tbgo9lkabnat2lo8Z ~]# ln -s /etc/centos-release sysinfo
[root@iZwz97tbgo9lkabnat2lo8Z ~]# ls -l sysinfo
lrwxrwxrwx 1 root root 19 Jun 20 19:21 sysinfo -> /etc/centos-release
[root@iZwz97tbgo9lkabnat2lo8Z ~]# cat sysinfo
CentOS Linux release 7.4.1708 (Core)
[root@iZwz97tbgo9lkabnat2lo8Z ~]# cat /etc/centos-release
CentOS Linux release 7.4.1708 (Core)
Copy the code
```
Description: Links can be divided into hard links and soft links (symbolic links). A hard link can be thought of as a pointer to file data, just like the reference count of objects in Python. Every time a hard link is added, the number of links in a file increases by 1. Only when the number of links in a file is 0, the storage space of the file can be overwritten by other files. We usually delete files actually does not delete the data on the hard disk, we just delete a pointer, or a use of data records, so similar to the “file disintegrator” software in “crush” file in addition to delete the file pointer, also in the file storage area to fill in the corresponding data to ensure that the file cannot return. A soft link is similar to a shortcut in Windows. If the linked file is deleted, the soft link becomes invalid.
Compress/uncompress and archive/unarchive – gzip/gunzip/xz.

Wget/root @ iZwz97tbgo9lkabnat2lo8Z ~ # http://download.redis.io/releases/redis-4.0.10.tar.gz -- 19:29:59-2018-06-20 http://download.redis.io/releases/redis-4.0.10.tar.gz Resolving download. Redis. IO (download. Redis. IO)... 109.74.203.151 Connecting to download. Redis. IO (download. Redis. IO) | 109.74.203.151 | : 80... connected. HTTP request sent, awaiting response... 200 OK Length: 1738465 (1.5m) [Application /x-gzip] Saving to: 'redis-4.0.0.tar.gz'100%[= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = >] 1738465 70.1 KB/sin 74s2018-06-20 19:31:14 (22.9 KB/s) - 'redis-4.0.0.tar.gz' saved [1738465/1738465] [root@iZwz97tbgo9lkabnat2lo8Z ~]# ls Redis * redis-4.0.0.tar. gz [root@iZwz97tbgo9lkabnat2lo8Z ~]# gunzip redis-4.0.0.tar. gz [root@iZwz97tbgo9lkabnat2lo8Z Ls ~] # redis * redis - 4.0.10. TarCopy the code

Archive and archive – tar.

[root@iZwz97tbgo9lkabnat2lo8Z ~]# tar-xvf redis-4.0.0.tar redis-4.0.10/ redis-4.0.10/.gitignore Redis-4.0.10/00-releasenotes Redis-4.0.10 /BUGS redis-4.0.10/CONTRIBUTING Redis-4.0.10 /COPYING redis-4.0.10/INSTALL MANIFESTO redis-4.0.10/MANIFESTO redis-4.0.10/Makefile Redis-4.0.10 / readme. md Redis-4.0.10 /deps/ redis-4.0.10/deps/Makefile Redis - 4.0.10 / deps/README. Md...Copy the code

Note: The tar command is used to create an archive. Generally, three parameters are required to create an archive: c indicates create, V indicates verbose, and f indicates the file to be specified. Unarchiving requires the -xvf parameter, where x stands for extract, and the other two parameters are the same as creating the archive.

Convert the standard input to the command line argument -xargs.

The following command will find the HTML files in the current path, and then pass these files as arguments to the rm command through xargs to find and delete the files.

[root@iZwz97tbgo9lkabnat2lo8Z ~]# find . -type f -name "*.html" | xargs rm -f
Copy the code

The following command converts the contents of multiple lines in the A.txt file into one line and outputs the contents to the B.txt file, where < indicates that input is read from a.txt and > indicates that command execution results are output to the B.txt file.

[root@iZwz97tbgo9lkabnat2lo8Z ~]# xargs < a.txt > b.txt
Copy the code

Note: This command is often used for piping (a means of implementing interprocess communication) and redirecting (reassigning input and output positions) operations, as demonstrated above, which will be described later.

Display a file or directory – basename/dirname.
Other related tools.

Sort – Sort the content
Uniq – Remove adjacent duplicate content
Tr – Replace the specified content with new content
Cut/paste – Cut/paste content
Split-split files
File – Determines the file type
Wc – Counts file lines, words, and bytes
Iconv – Encoding conversion

[root ~]# cat foo.txt grape apple pitaya [root ~]# cat bar.txt 100 200 300 400 [root ~]# paste foo.txt bar.txt grape 100  apple 200 pitaya 300 400 [root ~]# paste foo.txt bar.txt > hello.txt [root ~]# cut -b 4-8 hello.txt pe 10 le 20 aya 3 0  [root ~]# cat hello.txt | tr '\t' ',' grape,100 apple,200 pitaya,300 ,400 [root ~]# split -l 100 sohu.html hello [root ~]# wget https://www.baidu.com/img/bd_logo1.png [root ~]# file bd_logo1.png bd_logo1.png: PNG image data, 540 x 258, 8-bit colormap, non-interlaced [root ~]# wc sohu.html 2979 6355 212527 sohu.html [root ~]# wc -l sohu.html 2979 sohu.html [root ~]# wget  http://www.qq.com -O qq.html [root ~]# iconv -f gb2312 -t utf-8 qq.htmlCopy the code

Pipes and redirection

The use of pipes – |.

Example: Find the number of files in the current directory.
```
[root ~]# find ./ | wc -l
6152
Copy the code
```
Example: List the files and folders in the current path and give each item a number.
```
Ls/root ~ # | cat - n 1 dump. 2 mongo RDB - 3.6.5 Python 3-4 redis 3.6.5-3.2.11 5 redis. ConfCopy the code
```
Example: Find the total number of records in record.log that contain AAA, but not BBB
```
[root ~]# cat record.log | grep AAA | grep -v BBB | wc -l
Copy the code
```

Output redirection and error redirect – > / >> / 2>.

[root ~]# cat readme.txt
banana
apple
grape
apple
grape
watermelon
pear
pitaya
[root ~]# cat readme.txt | sort | uniq > result.txt
[root ~]# cat result.txt
apple
banana
grape
pear
pitaya
watermelon
Copy the code

Input redirection – <.

[root ~]# echo 'hello, world! ' > hello.txt [root ~]# wall < hello.txt [root ~]# Broadcast message from root (Wed Jun 20 19:43:05 2018): hello, world! [root ~]# echo 'I will show you some code.' >> hello.txt [root ~]# wall < hello.txt [root ~]# Broadcast message from root (Wed Jun 20 19:43:55 2018): hello, world! I will show you some code.Copy the code

Multiple orientation – tee.

In addition to displaying the result of the command ls on the terminal, the following command is appended to the ls.txt file.
```
[root ~]# ls | tee -a ls.txt
Copy the code
```

The alias

alias

[root ~]# alias ll='ls -l'
[root ~]# alias frm='rm -rf'
[root ~]# ll
...
drwxr-xr-x  2 root       root   4096 Jun 20 12:52 abc
...
[root ~]# frm abc
Copy the code

unalias

[root ~]# unalias frm
[root ~]# frm sohu.html
-bash: frm: command not found
Copy the code

Text processing

Character stream editor – sed.

Sed is a tool for manipulating, filtering, and transforming text content. Suppose you have a file named fruit.txt with the following contents.
```
[root ~]# cat -n fruit.txt 
     1  banana
     2  grape
     3  apple
     4  watermelon
     5  orange
Copy the code
```
Next, we add a pitaya after line 2.
```
[root ~]# sed '2a pitaya' fruit.txt 
banana
grape
pitaya
apple
watermelon
orange
Copy the code
```
Note: This command, like many of our previous commands, does not change the fruit.txt file. Instead, it outputs the content with a new line to the terminal. If you want to save to fruit.txt, you can use the output redirection operation.

Insert a Waxberry before line 2.
```
[root ~]# sed '2i waxberry' fruit.txt
banana
waxberry
grape
apple
watermelon
orange
Copy the code
```
Delete line 3.
```
[root ~]# sed '3d' fruit.txt
banana
grape
watermelon
orange
Copy the code
```
Delete lines 2 through 4.
```
[root ~]# sed '2,4d' fruitCopy the code
```
Replace the character a in the text with @.
```
[root ~]# sed 's#a#@#' fruit.txt 
b@nana
gr@pe
@pple
w@termelon
or@nge
Copy the code
```
Replace the character a in the text with @, using global mode.
```
[root ~]# sed 's#a#@#g' fruit.txt 
b@n@n@
gr@pe
@pple
w@termelon
or@nge
Copy the code
```
Pattern matching and processing language – AWK.

Awk is a programming language and the most powerful tool for processing text in Linux. One of its authors and its current maintainer is the aforementioned Brian Kernighan (Ken and DMR’s closest associate). This command is powerful enough to extract the specified columns from text, use regular expressions to extract what we want from text, display the specified rows, and perform statistics and calculations.

Suppose you have a file named fruit2.txt with the following contents.
```
[root ~]# cat fruit2.txt 
1       banana      120
2       grape       500
3       apple       1230
4       watermelon  80
5       orange      400
Copy the code
```
Display line 3 of the file.
```
[root ~]# awk 'NR==3' fruit2.txt 
3       apple       1230
Copy the code
```
Displays column 2 of the file.
```
[root ~]# awk '{print $2}' fruit2.txt 
banana
grape
apple
watermelon
orange
Copy the code
```
Displays the last column of the file.
```
[root ~]# awk '{print $NF}' fruit2.txt 
120
500
1230
80
400
Copy the code
```
Output lines with trailing digits greater than or equal to 300.
```
[root ~]# awk '{if($3 >= 300) {print $0}}' fruit2.txt 
2       grape       500
3       apple       1230
5       orange      400
Copy the code
```
What is shown above is just the tip of the awK command iceberg, much more is left to the reader to explore in practice.

User management

Create and delete users – useradd/userdel.
```
[root home]# useradd hellokitty
[root home]# userdel hellokitty
Copy the code
```
- -d– When creating a user, specify the home directory for the user
- -g– When creating a user, specify the user group to which the user belongs
Create and delete user groups – groupadd/groupdel.

Note: User groups are used to facilitate the management of all users in a group.
Change the password – passwd.
```
[root ~]# passwd hellokitty
New password: 
Retype new password: 
passwd: all authentication tokens updated successfully.
Copy the code
```
Note: The input password and confirm password must be the same as the confirm password. The input password must be the same as the confirm password. If no object is specified by the passwd command, the password of the current user is to be changed. To change user passwords in batches, run the chpasswd command.
- -l / -u– Locks or unlocks a user.
- -d– Clear the user password.
- -e– Set the password to expire immediately. Users are required to change their passwords upon login.
- -i– Set the number of days after the password expires to disable the user.
View and change the password validity period – chage.

A hellokitty user must change its password after 100 days. The user will be notified 15 days before expiration, and the user will be disabled 15 days after expiration.
```
chage -M 100 -W 15 -I 15 hellokitty
Copy the code
```

Switch the user to su.

[root ~]# su hellokitty
[hellokitty root]$
Copy the code

Run the -sudo command as administrator.
```
[hellokitty ~]$ ls /root
ls: cannot open directory /root: Permission denied
[hellokitty ~]$ sudo ls /root
[sudo] password for hellokitty:
Copy the code
```
Note: If you want the user to be able to execute commands as an administrator, the user must appear in the sudoers list. The sudoers file is in /etc. If you want to edit the file directly, you can also use the following command.

Edit sudoers file – visudo.

The editor used here is vi, which is explained later. Part of the file looks like this:

## Allow root to run any commands anywhere 
root    ALL=(ALL)   ALL

## Allows members of the 'sys' group to run networking, software, 
## service management apps and more.
# %sys ALL = NETWORKING, SOFTWARE, SERVICES, STORAGE, DELEGATING, PROCESSES, LOCATE, DRIVERS
## Allows people in group wheel to run all commands
%wheel  ALL=(ALL)   ALL

## Same thing without a password
# %wheel ALL=(ALL) NOPASSWD: ALL

## Allows members of the users group to mount and unmount the
## cdrom as root
# %users ALL=/sbin/mount /mnt/cdrom, /sbin/umount /mnt/cdrom

## Allows members of the users group to shutdown this system
# %users localhost=/sbin/shutdown -h now
Copy the code

– ID displays information about a user and a user group.

Send messages to other users -write/wall.

The sender:

[root ~]# write hellokitty
Dinner is on me.
Call me at 6pm.
Copy the code

Receiver:

[hellokitty ~]$ 
Message from root on pts/0 at 17:41 ...
Dinner is on me.
Call me at 6pm.
EOF
Copy the code

View/Set whether to receive messages from other users – mesg.

[hellokitty ~]$ mesg
is y
[hellokitty ~]$ mesg n
[hellokitty ~]$ mesg
is n
Copy the code

The file system

Files and Paths

Naming rules: The maximum length of a file name depends on the type of the file system. Generally, a file name cannot exceed 255 characters. Although most characters can be used in a file name, it is best to use symbols such as uppercase and lowercase letters, digits, underscores (_), and periods (.). You can use Spaces in file names, but you should avoid them as much as possible. Otherwise, you will need to put the file name in double quotation marks or pass the file name\Escape whitespace.
Extension: File extensions are optional on Linux, but using extensions helps you understand the contents of the file. Some applications identify files by extension, but many do not rely on file extensions, as infileThe command does not identify the file type based on the extension.
Hidden files: Files that begin with a dot are hidden files (invisible files) on Linux.

The directory structure

/bin – Binary file of basic commands.
/boot – Static file for the boot loader.
/dev/device file.
/etc/configuration file.
/home – The parent directory of the home directory for common users.
/lib – Shared library file.
/lib64 – Share 64-bit library files.
/ Lost +found – Store unlinked files.
/media – Automatically identifies the mount directory of the device.
/ MNT – Mount point for a temporarily mounted file system.
/opt – Optional installation position of the plug-in software package.
/proc – Kernel and process information.
/root – Primary directory for the super administrator.
/run – Stores things that the system needs to run.
/sbin – The binary file of the superuser.
/sys – Pseudo file system for the device.
/ TMP – Temporary folder.
/usr – User application directory.
/var – Variable data directory.

Access permissions

Chmod – Changes file mode bits.
```
[root ~]# ls -l
...
-rw-r--r--  1 root       root 211878 Jun 19 16:06 sohu.html
...
[root ~]# chmod g+w,o+w sohu.html
[root ~]# ls -l
...
-rw-rw-rw-  1 root       root 211878 Jun 19 16:06 sohu.html
...
[root ~]# chmod 644 sohu.html
[root ~]# ls -l
...
-rw-r--r--  1 root       root 211878 Jun 19 16:06 sohu.html
...
Copy the code
```
Through the above example can be seen, with chmod to change the file mode than special two ways: one is the character setting method, the other is the number setting method. In addition to chmod, you can use umask to set which permissions will be removed from the default permissions for new files.

The following table lists the results and permissions for viewing directories or files in long format.

Chown – Changes the file owner.

[root ~]# ls -l
...
-rw-r--r--  1 root root     54 Jun 20 10:06 readme.txt
...
[root ~]# chown hellokitty readme.txt
[root ~]# ls -l
...
-rw-r--r--  1 hellokitty root     54 Jun 20 10:06 readme.txt
...
Copy the code

CHGRP – Changes the user group.

Disk management

Lists the disk usage of the file system – df.

[root ~]# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1        40G  5.0G   33G  14% /
devtmpfs        486M     0  486M   0% /dev
tmpfs           497M     0  497M   0% /dev/shm
tmpfs           497M  356K  496M   1% /run
tmpfs           497M     0  497M   0% /sys/fs/cgroup
tmpfs           100M     0  100M   0% /run/user/0
Copy the code

Disk partition table operation – fdisk.

[root ~]# fdisk -l Disk /dev/vda: 42.9 GB, 42949672960 bytes, 83886080 sectors Units = sectors of 1 * 512 = 512 Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x000a42f4 Device Boot Start End Blocks Id System /dev/vda1 * 2048 83884031 41940992 83 Linux Disk /dev/vdb: 21.5 GB, 21474836480 bytes, 41943040 sectors Units = sectors of 1 * 512 = 512 Bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytesCopy the code

Disk partitioning tool – Parted.
Format the file system – MKFS.
```
[root ~]# mkfs -t ext4 -v /dev/sdb
Copy the code
```
- -t– Specify the type of the file system.
- -c– Check disk damage when creating a file system.
- -v– Displays detailed information.
File system check – FSCK.
Convert or copy files – dd.
Mount/Unmount – mount/umount.
Create/activate/disable swap partitions -mkswap/swapon/swapoff.

Note: Executing the above commands is risky. If you do not know how to use these commands, you are advised not to use them at will. During the use of these commands, you are advised to refer to the reference materials for operation and confirm whether to do so before operation.

Editor – Vim

Start the vim. Vim can be started by using the vi or vim command. A file can be opened by specifying a file name when it is started, or by specifying a file name when it is saved.
```
[root ~]# vim guess.py
Copy the code
```
Command mode, Edit mode, and last line mode: Vim is started to enter command mode (also known as Normal mode). If you enter the English letter I in command mode, you will enter the edit mode (Insert mode). The — Insert — prompt appears at the bottom of the screen. In edit mode, press Esc to return to command mode. At this point, if you type:, the last line mode will be entered. In the last line mode, type Q! You can forcibly exit Vim without saving your current work; Typing v in command mode will bring you into Visual mode (Visual mode), where you can select a region with the cursor to complete the corresponding operation.
Save and exit vim: Enter the last line mode in command mode, and enter wq to save and exit. To discard the edited content, type Q! Forcible withdrawal, which has already been mentioned; In command mode, you can also enter ZZ to save and exit. If you want to save the file without exiting, type W in last-line mode; You can specify the file name to save by typing a space after w.
Cursor manipulation.
- Yes In command modeh,j,k,lTo control the cursor to move left, down, up and right, you can enter a number before a letter to indicate the distance moved, for example:10hRepresents a move of 10 characters to the left.
- Yes In command modeCtrl+yandCtrl+eTo achieve the operation of scrolling up and down a line of text, can be passedCtrl+fandCtrl+bTo page forward and backward.
- You can enter English letters in command modeGMove the cursor to the end of the fileggMove the cursor to the beginning of the fileGEnter a number to move the cursor to the specified line.
Text manipulation.
- Delete: Available in command modeddTo delete the entire line; Can be found inddPrefixes a number to specify the number of rows to delete; You can used$To delete the operation from the cursor to the end of the lined0To delete from the cursor to the beginning of the line; If you want to delete a word, you can usedw; If you want to delete the full text, you can enter:%d(including:Used to enter last-line mode from command mode).
- Copy and paste: this works in command modeyyTo copy the entire line; Can be found inyyPrefixes a number to specify the number of lines to copy; Can be achieved bypPaste the copied content where the cursor is.
- Undo and Restore: Enter in command modeuYou can undo previous actions; throughCtrl+rYou can restore an action that has been undone.
- Sort content: Enter in command mode%! sort.
Find and replace.
- The lookup operation requires input/Enter last-line mode and provide a regular expression to match the corresponding content, for example:/doc.*\., the inputnTo search forward, you can also typeNSearch backwards.
- The replacement operation requires input:Enter last-line mode and specify the search scope, regular expression, and replacement content and matching options, for example::1,$s/doc.*/hello/gice, including:
  - g– global: indicates a global match.
  - i– Ignore case: case matching is ignored.
  - c– confirm: The replacement must be confirmed.
  - e– error: ignores errors.
Parameter setting: ViM can be set after input: enter the last line mode.
- Set ts=4
- To set the line number to show/not to show: set nu/set nonu
- To set highlighting on/off: syntax on/syntax off
- Set the display ruler (row and column where the cursor is) : Set ruler
- Set to enable/disable search result highlighting: set HLS/set nohls
  
  Note: If you want the above Settings to automatically take effect each time you start Vim, you need to write these Settings to the. Vimrc file in the user’s home directory.
Advanced techniques
- Compare multiple files.
```
[root ~]# vim -d foo.txt bar.txt
Copy the code
```
- Open multiple files.
```
[root ~]# vim foo.txt bar.txt hello.txt
Copy the code
```
  After vim is started, only one window displays foo. TXT. You can enter ls in the last line mode to view the three open files, or enter B
  
  in the last line mode to display another file, for example, b 2 to display bar. TXT, b 3 to display hello. TXT.
- Split and switch Windows.
  
  We can split the Windows horizontally or vertically by typing SP or VS in the last line mode, so that we can open multiple edit Windows at the same time. We can switch edit Windows by pressing Ctrl+ W twice. Exit in one window will close the corresponding window, and the other Windows will remain.

Mapping shortcut keys: ViM allows you to map some common operations to shortcut keys to improve work efficiency.
- Example 1: Enter F4 in command mode to delete 10000 lines of code from the first line.
  
  : the map (F4 > gg10000dd.
  
  Example 2: in the edit mode input __main directly completion as if __name__ = = “__main__ ‘:.
  
  :inoremap __main if __name__ == '__main__':
Note: The I in the InoreMap of Example 2 above means that the mapped key is used in edit mode, and nore means that it is important not to recurse, otherwise if the key itself appears in the content corresponding to the key, it will cause recursion (equivalent to an infinite loop). If you want the mapping shortcut to take effect every time you start vim, you need to write the mapping to the. Vimrc file in the user’s home directory.
Recording macros.
- In command mode, type QA to start recording the macro (where a is the register name, or any other English letter or number from 0 to 9).
- Perform your actions (cursor actions, edit actions, etc.) and these actions will be recorded.
- If the recording operation is complete, press Q to end the recording.
- Play the macro through @a (a is the name of the register you just used), or prefix it with a number if you want to execute the macro more than once, such as 100@a to play the macro 100 times.
- To get a taste of recording macros, try the following example from Harttle Land, which provides tips on using vim.

Software installation and configuration

Use package management tools

yum– Yellowdog Updater Modified.
- yum search: Searches for software packages, for exampleyum search nginx.
- yum list installed: Lists the installed software packages, for exampleyum list installed | grep zlib.
- yum install: Installs the software package, for exampleyum install nginx.
- yum remove: Deletes a software package, for exampleyum remove nginx.
- yum update: Updates the software package, for exampleyum updateCan update all packages, whileyum update tarOnly tar will be updated.
- yum check-update: Check which packages are available for updating.
- yum info: Displays information about software packages, for exampleyum info nginx.
rpm– Redhat Package Manager.
- Installation package:rpm -ivh <packagename>.rpm.
- Removing a software package:rpm -e <packagename>.
- Querying software packages:rpm -qaFor example, you can userpm -qa | grep mysqlTo check whether MySQL related packages are installed.

The following uses Nginx as an example to demonstrate how to install software using Yum.

[root ~]# yum -y install nginx ... El7 Dependency Installed: nginx.x86_64 1:1.12.2-2.el7 Dependency Installed: El7 nginx-mod-http-geoip.x86_64 1:1.12.2-2.el7 nginx-mod-http-image-filter.x86_64 El7 nginx-mod-http-perl-x86_64 1:1.12.2-2.el7 nginx-mod-http-XSLt-filter.x86_64 1:1.12.2-2.el7 X86_64 1:1.12.2-2.el7 nginx-mod-stream.x86_64 1:1.12.2-2.el7 Complete! [root ~]# yum info nginx Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile Installed Packages Name : nginx Arch : x86_64 Epoch : 1 Version: 1.12.2 Release: 2.el7 Size: 1.5m Repo: installed From Repo: epel Summary: A high performance web server and reverse proxy server URL : http://nginx.org/ License : BSD Description : Nginx is a web server and a reverse proxy server for HTTP, SMTP, POP3 and : IMAP protocols, with a strong focus on high concurrency, performance and low : Memory usage. [root ~]# nginx -v nginx version: nginx/1.12.2Copy the code

Remove the Nginx.

[root ~]# yum -y remove nginx
Copy the code

The following uses MySQL as an example to demonstrate how to install the software using RPM. To install MySQL, go to the MySQL official website and download the RPM file. Of course, select the version of Linux you are using. MySQL is now a product of Oracle Corporation. After MySQL was acquired, the authors of MySQL created a new branch of MySQL, MariaDB, which can be installed through YUM.

[root mysql]# ls mysql-community-client-5.7.22-1.el7.x86_64. RPM mysql-community-common-5.7.22-1.el7.x86_64. RPM RPM [root mysql]# yum -y remove mysql-community-libs-5.7.22-1.el7.x86_64. RPM [root mysql]# yum -y remove mysql-community-libs-5.7.22-1.el7.x86_64. RPM [root mysql]# yum -y remove mysql-community-libs-5.7.22-1.el7.x86_64. RPM Mariadb -libs [root mysql]# yum -y install libaio [root mysql]# RPM -ivh mysql-community-common-5.7.26-1.el7.x86_64. RPM . [root mysql]# RPM -ivh mysql-community-libs-5.7.26-1.el7.x86_64. [root mysql]# RPM -ivh mysql-community-client-5.7.26-1.el7.x86_64. [root mysql]# RPM -ivh mysql-community-server-5.7.26-1.el7.x86_64.Copy the code

Note: due to the MySQL and MariaDB depends on the bottom of the library is a conflict, so the above we first use yum removed called MariaDB – libs dependent libraries and installed called libaio dependent libraries support asynchronous I/O operations. For information on the relationship between MySQL and MariaDB, you can read about MariaDB on Wikipedia.

Remove the installed MySQL.

[root ~]# rpm -qa | grep mysql | xargs rpm -e
Copy the code

Download decompression configuration environment variables

The following uses installing MongoDB as an example to demonstrate how such software should be installed.

Wget/root ~ # https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel70-3.6.5.tgz -- 18:32:53-2018-06-21 https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel70-3.6.5.tgz Resolving fastdl.mongodb.org (fastdl.mongodb.org)... 52.85.83.16, 52.85.83.228, 52.85.83.186,... Connecting to fastdl.mongodb.org (fastdl.mongodb.org) | 52.85.83.16 | : 443... connected. HTTP request sent, awaiting response... 200 OK Length: 100564462 (96M) [Application /x-gzip] Saving to: 'mongodb-linux-x86_64-rhel70-3.6.5.tgz'100%[= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = >] 100564462 630 KB/sin 2m 9s2018-06-21 18:35:04 (76KB /s) - 'mongodb-linux-x86_64-rhel70-3.6.5.tgz' saved [100564462/100564462] [root ~]# gunzip [root ~]# tar mongodb-linux-x86_64-rhel70-3.6.5.tar [root ~]# tar -xvf mongodb-linux-x86_64-rhel70-3.6.5.tar Mongo - Linux - x86_64 - rhel70 3.6.5 / README mongo - Linux - x86_64 - rhel70-3.6.5 / THIRD - PARTY - NOTICES Mongo - Linux - x86_64 - rhel70-3.6.5 / mongo MPL - 2 - Linux - x86_64 - rhel70-3.6.5 / GNU AGPL - 3.0 Mongo - Linux - x86_64 - rhel70 3.6.5 / bin/mongodump mongo - Linux - x86_64 - rhel70 3.6.5 / bin/mongorestore Mongo - Linux - x86_64 - rhel70 3.6.5 / bin/mongoexport mongo - Linux - x86_64 - rhel70 3.6.5 / bin/mongoimport Mongo - Linux - x86_64 - rhel70 3.6.5 / bin/mongostat mongo - Linux - x86_64 - rhel70 3.6.5 / bin/mongotop Mongo - Linux - x86_64 - rhel70 3.6.5 / bin/bsondump mongo - Linux - x86_64 - rhel70 3.6.5 / bin/mongofiles Mongo - Linux - x86_64 - rhel70 3.6.5 / bin/mongoreplay mongo - Linux - x86_64 - rhel70 3.6.5 / bin/mongoperf Mongo - Linux - x86_64 - rhel70 3.6.5 / bin/mongod mongo - Linux - x86_64 - rhel70 3.6.5 / bin/mongos Mongodb -linux-x86_64-rhel70-3.6.5/bin/mongo mongodb-linux-x86_64-rhel70-3.6.5/bin/install_compass [root ~]# vim .bash_profile ... PATH = $PATH: $HOME/bin: $HOME/mongo - Linux - x86_64 rhel70-3.6.5 / bin export PATH... [root ~]# mongod -- db version v3.6.5 git version: A20ecd3e3a174162052ff99913bc2ca9a839d618 OpenSSL version: OpenSSL 1.0.1 e - 11 Feb 2013 allocator: fips tcmalloc modules: none build environment: distmod: rhel70 distarch: x86_64 target_arch: X86_64 [root ~]# MongoDB shell version v3.6.5 git version: A20ecd3e3a174162052ff99913bc2ca9a839d618 OpenSSL version: OpenSSL 1.0.1 e - 11 Feb 2013 allocator: fips tcmalloc modules: none build environment: distmod: rhel70 distarch: x86_64 target_arch: x86_64Copy the code

Note: You can also install MongoDB using Yum. For details, see the instructions on the official website.

Source code build installation

Install Python 3.6.

/ root ~ # yum install GCC/root ~ # wget HTTP: / / https://www.python.org/ftp/python/3.6.5/Python-3.6.5.tgz/root ~ # gunzip TGZ [root ~]# tar -xvf python-3.6.5.tar [root ~]# CD python-3.6.5 [root ~]#./configure --prefix=/usr/local/python36 --enable-optimizations [root ~]# yum -y install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gdbm-devel db4-devel libpcap-devel xz-devel [root ~]# make && make install ... / root ~ # ln -s/usr/local/python36 / bin/python3.6 / usr/bin/python3 / root ~ # python3 -- version Python 3.6.5 root ~ # python3 -m pip install -U pip [root ~]# pip3 --versionCopy the code

Note: You need to register the PATH environment variable after installing Python. You need to register the absolute PATH of the bin folder in the Python installation PATH into the PATH environment variable. Registering environment variables can modify profile files in the user’s home directory. Bash_profile or /etc. The difference is that the former is equivalent to user environment variables, while the latter is equivalent to system environment variables.

Install Redis – 3.2.12.

/ root ~ # wget HTTP: / / http://download.redis.io/releases/redis-3.2.12.tar.gz/root ~ # gunzip redis - 3.2.12. Tar. Gz/root ~ # tar -xvf redis-3.2.12.tar [root ~]# CD redis-3.2.12 [root ~]# make && make install [root ~]# redis-server --version redis Server v=3.2.12 SHA =00000000:0 malloc= jemalloc-4.0.3bits =64 build= 5bC5CD3C03d6ceb6 [root ~]# redis-cli --version Redis - cli 3.2.12Copy the code

Configure the service

We can install and configure services on Linux, which means we can make Linux a database server, a Web server, a cache server, a file server, a message queue server, and so on. Most services under Linux are set up as daemons (which reside in the background but do not stop Linux from running because the service is still running), so we usually install services with a d after their name, which is short for the English word daemon, for example: The firewall service is called Firewalld, the MySQL service we installed before is called mysqld, the Apache server is called HTTPD, etc. After the service is installed, you can run the systemctl command or service command to start or stop the service.

Start the firewall service.

[root ~]# systemctl start firewalld
Copy the code

The firewall service is terminated.

[root ~]# systemctl stop firewalld
Copy the code

Restart the firewall service.

[root ~]# systemctl restart firewalld
Copy the code

Check the firewall service status.

[root ~]# systemctl status firewalld
Copy the code

Example Set or disable the firewall service startup.

[root ~]# systemctl enable firewalld
Created symlink from /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service to /usr/lib/systemd/system/firewalld.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/firewalld.service to /usr/lib/systemd/system/firewalld.service.
[root ~]# systemctl disable firewalld
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
Copy the code

Schedule a task

Executes the command at the specified time
- At – Queues tasks to be executed at a specified time.
- Atq – Looks up the queue of tasks to be executed.
- Atrm – Deletes tasks to be executed from the queue.
Specify tasks to be performed 3 days from now at 5pm.
```
[root ~]# at 5pm+3days
at> rm -f /root/*.html
at> <EOT>
job 9 at Wed Jun  5 17:00:00 2019
Copy the code
```
Look up the queue of tasks to be executed.
```
[root ~]# atq
9       Wed Jun  5 17:00:00 2019 a root
Copy the code
```
Removes the specified task from the queue.
```
[root ~]$ atrm 9
Copy the code
```

Scheduled task table – crontab.

[root ~]# crontab -e
* * * * * echo "hello, world!" >> /root/hello.txt
59 23 * * * rm -f /root/*.log
Copy the code

Note: The crontab -e command will open vim to edit Cron expressions and specify tasks to trigger. The first scheduled task is to append hello, world! ; The other is to delete files with the log suffix in the /root directory at 23:59 every day. If you don’t know how to write Cron expressions, follow the tips in the /etc/crontab file (covered below) or use a search engine to find “Cron Expression online generator” to generate Cron expressions.

You can also customize scheduled tasks by modifying the crontab file in the /etc//etc/directory.

[root ~]# cd /etc [root etc]# ls -l | grep cron -rw-------. 1 root root 541 Aug 3 2017 anacrontab drwxr-xr-x. 2 root root 4096 Mar 27 11:56 cron.d drwxr-xr-x. 2 root root 4096 Mar 27 11:51 cron.daily -rw-------. 1 root root 0 Aug 3 2017 cron.deny drwxr-xr-x. 2 root root 4096 Mar 27 11:50 cron.hourly drwxr-xr-x. 2 root root 4096 Jun 10 2014 cron.monthly -rw-r--r-- 1 root root 493 Jun 23 15:09 crontab drwxr-xr-x. 2 root root 4096 Jun 10 2014 cron.weekly [root etc]# vim crontab 1 SHELL=/bin/bash 2 PATH=/sbin:/bin:/usr/sbin:/usr/bin 3 MAILTO=root 4 5 # For details see man 4 crontabs 6 7 # Example of job definition: 8 # .---------------- minute (0 - 59) 9 # | .------------- hour (0 - 23) 10 # | | .---------- day of month (1 - 31) 11 #  | | | .------- month (1 - 12) OR jan,feb,mar,apr ... 12 # | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat 13 # | | | | | 14 # * * * * * user-name command to be executedCopy the code

Network access and management

Secure Remote Connection – SSH.

[root ~]$SSH [email protected] The authenticity of host '120.77.222.217 (120.77.222.217)' can't be established  key fingerprint is SHA256:BhUhykv+FvnIL03I9cLRpWpaCxI91m9n7zBWrcXRa8w. ECDSA key fingerprint is MD5:cc:85:e9:f0:d7:07:1a:26:41:92:77:6b:7f:a0:92:65. Are you sure you want to continue connecting (yes/no)? Yes Warning: Permanently added '120.77.222.217' (ECDSA) to the list of known hosts. [email protected]'s password:Copy the code

Obtaining resources from the network – wget.
- -b Indicates the background download mode
- -o Downloads the file to the specified directory
- -r Recursive download
Send and receive mail – mail.

Network Configuration Tool (old) – ifconfig.

[root ~]# ifconfig eth0 eth0: Flags = 4163 < UP, BROADCAST, RUNNING, MULTICAST > mtu 1500 inet 172.18.61.250 netmask 255.255.240.0 BROADCAST 172.18.63.255 Ether00:16:3e :02:b6:46 TXQueuelen 1000 (Ethernet) RX packets 1067841 bytes 1296732947 (1.2 GiB) RX errors 0 Dropped 0 Overruns 0 frame 0 TX packets 409912 bytes 43569163 (41.5 MiB) TX errors 0 Dropped 0 Overruns 0 carrier 0 collisionsCopy the code

Network Configuration Tool (new) – IP.

[root ~]# ip address 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host LO valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:16:3e:02:b6:46 brd Ff :ff:ff:ff:ff:ff :ff inet 172.18.61.250/20 BRD 172.18.63.255 scope global eth0 valid_lft forever preferred_lft foreverCopy the code

Network reachability check – ping.

[root ~]# ping www.baidu.com -c 3 ping www.a.shifen.com (220.181.111.188) 56(84) bytes of data.64 bytes from 220.181.111.188 (220.181.111.188): icmp_seq=1 TTL =51 time=36.3 ms 64 bytes from 220.181.111.188 (220.181.111.188): Icmp_seq =2 TTL =51 time=36.4 ms 64 bytes from 220.181.111.188 (220.181.111.188): Icmp_seq =3 TTL =51 time=36.4 ms -- www.a.shifen.com ping statistics -- 3 packets transmitted, 3 received, 0% packet loss, time 2002 RTT min/avg/Max/ms mdev 36.427/0.156 = 36.392/36.406 / msCopy the code

Displays or manages the routing table – route.

View network services and ports – netstat/ss.

[root ~]# netstat -nap | grep nginx
Copy the code

Network listening packet capture – tcpdump.
Secure file copy – SCP.

/ root ~ # SCP [email protected]: / root/guido. JPG [email protected]: / home/hellokitty/PIC. JPGCopy the code

File synchronization tool – rsync.

Note: Using Rsync can achieve automatic file synchronization, which is very important for the file server. The usage of this command will be explained later when we talk about project deployment.
Secure File Transfer – SFTP.
```
[root ~]# sftp [email protected]
[email protected]'s password:
Connected to 1.2.3.4.
sftp>
Copy the code
```
- Help: Displays the help information.
- Ls/LLS: displays the list of remote and local directories.
- CD/LCD: Switches the remote or local path.
- Mkdir /lmkdir: create a remote/local directory.
- PWD/LPWD: displays the current remote/local working directory.
- Get: Downloads the file.
- Put: uploads files.
- Rm: Delete the remote file.
- Bye /exit/quit: Exits SFTP.

Process management

View the process -ps.

[root ~]# ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 Jun23 ?        00:00:05 /usr/lib/systemd/systemd --switched-root --system --deserialize 21
root         2     0  0 Jun23 ?        00:00:00 [kthreadd]
...
[root ~]# ps -ef | grep mysqld
root      4943  4581  0 22:45 pts/0    00:00:00 grep --color=auto mysqld
mysql    25257     1  0 Jun25 ?        00:00:39 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid
Copy the code

The process status tree – Pstree is displayed.

[root ~]# PstREE Systemd ─┬─ ├─ 12 *[{AliYunDun}] ├─ ─ 12 *[{AliYunDun}] ├─ 12 *[{AliYunDun}] ├─ 12 *[{AliYunDun}] ├─ 12 *[{AliYunDun}] ├ ─ aliyun - service ─ ─ ─ 2 * [{aliyun - service}] ├ ─ atd ├ ─ auditd ─ ─ ─ {auditd} ├ ─ dbus - daemon ├ ─ dhclient ├ ─ called irqbalance ├ ─ lvmetad 28 * ├ ─ mysqld ─ ─ ─ [{mysqld}] ├ ─ nginx ─ ─ ─ 2 * / nginx ├ ─ NTPD ├ ─ polkitd ─ ─ ─ 6 * [{polkitd}] ├ ─ rsyslogd ─ ─ ─ 2 * [{rsyslogd}] ├─ ├─ Bass Exercises ├─ Bass Exercises ├─ Bass Exercises ├─ Bass Exercises ├─ Bass Exercises ├─ Bass Exercises ├─ Bass Exercises ├─ Bass Exercises ├─ Bass Exercises ├─ Proc.Copy the code

Find the process matching the specified condition – pgrep.
```
[root ~]$ pgrep mysqld
3584
Copy the code
```

Terminating a process by process ID – kill.

[root ~]$ kill -l
 1) SIGHUP       2) SIGINT       3) SIGQUIT      4) SIGILL       5) SIGTRAP
 6) SIGABRT      7) SIGBUS       8) SIGFPE       9) SIGKILL     10) SIGUSR1
11) SIGSEGV     12) SIGUSR2     13) SIGPIPE     14) SIGALRM     15) SIGTERM
16) SIGSTKFLT   17) SIGCHLD     18) SIGCONT     19) SIGSTOP     20) SIGTSTP
21) SIGTTIN     22) SIGTTOU     23) SIGURG      24) SIGXCPU     25) SIGXFSZ
26) SIGVTALRM   27) SIGPROF     28) SIGWINCH    29) SIGIO       30) SIGPWR
31) SIGSYS      34) SIGRTMIN    35) SIGRTMIN+1  36) SIGRTMIN+2  37) SIGRTMIN+3
38) SIGRTMIN+4  39) SIGRTMIN+5  40) SIGRTMIN+6  41) SIGRTMIN+7  42) SIGRTMIN+8
43) SIGRTMIN+9  44) SIGRTMIN+10 45) SIGRTMIN+11 46) SIGRTMIN+12 47) SIGRTMIN+13
48) SIGRTMIN+14 49) SIGRTMIN+15 50) SIGRTMAX-14 51) SIGRTMAX-13 52) SIGRTMAX-12
53) SIGRTMAX-11 54) SIGRTMAX-10 55) SIGRTMAX-9  56) SIGRTMAX-8  57) SIGRTMAX-7
58) SIGRTMAX-6  59) SIGRTMAX-5  60) SIGRTMAX-4  61) SIGRTMAX-3  62) SIGRTMAX-2
63) SIGRTMAX-1  64) SIGRTMAX
[root ~]# kill 1234
[root ~]# kill -9 1234
Copy the code

Example: Use a command to forcibly terminate a running Redis process.

ps -ef | grep redis | grep -v grep | awk '{print $2}' | xargs kill
Copy the code

Killing a process by its name – killall/pkill.

End the process named mysqld.
```
[root ~]# pkill mysqld
Copy the code
```
End all processes for a hellokitty user.
```
[root ~]# pkill -u hellokitty
Copy the code
```
Note: this action will disconnect the hellokitty user from the server.
Run the process in the background.
- Ctrl+Z– Shortcut keys to stop the process and put it in the background.
- &– Run the process in the background.
```
[root ~]# mongod &
[root ~]# redis-server
...
^Z
[4]+  Stopped                 redis-server
Copy the code
```

Example Query the background process -jobs.

[root ~]# jobs
[2]   Running                 mongod &
[3]-  Stopped                 cat
[4]+  Stopped                 redis-server
Copy the code

Let the process continue running in the background – bg.

[root ~]# bg %4
[4]+ redis-server &
[root ~]# jobs
[2]   Running                 mongod &
[3]+  Stopped                 cat
[4]-  Running                 redis-server &
Copy the code

Put background process in foreground -fg.
```
[root ~]# fg %4
redis-server
Copy the code
```
Note: A foreground process can be terminated with Ctrl+C.
Adjust program/process runtime priority – nice/Renice.

The process continues to work after the user logs out – nohup.

[root ~]# nohup ping www.baidu.com > result.txt &
Copy the code

Track process system calls – strace.

[root ~]# pgrep mysqld
8803
[root ~]# strace -c -p 8803
strace: Process 8803 attached
^Cstrace: Process 8803 detached
%time seconds usecs/call calls errors syscall-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 5719 1 restart_syscall 0.49 0.005719 99.18 0.000028 46 1 Clone 0.05 0.000003 3 1 Mmap 0.03 0.000002 2 1 Accept ------ ----------- ----------- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 100.00 0.005766 5 totalCopy the code

Note: The usage and parameters of this command are complex. You are advised to learn about this command based on actual requirements.

View the current runlevel – runlevel.
```
[root ~]# runlevel
N 3
Copy the code
```

Monitoring the resource usage of processes in real time – top

[root ~]# user, load Average: 0.00, 0.01, 0.05 Tasks: 65 total, 1 running, 64 sleeping, 0 stopped, 0 zombie%Cpu(S): 0.3US, 0.3SY, 0.0Ni, 99.3ID, 0.0wa, 0.0HI, 0.0Si, 0.0ST
KiB Mem :  1016168 total,   191060 free,   324700 used,   500408 buff/cache
KiB Swap:        0 total,        0 free,        0 used.   530944 avail Mem
...
Copy the code

-c– Displays the entire process path.
-d– Specifies the interval (in seconds) between two brushes.
-i– Do not display idle processes or zombie processes.
-p– Displays information about a specified process.

System in the diagnosis of

System startup exception diagnosis – dmesg.

View system activity information – SAR.

[root ~]# sar-u-r 5 10 Linux 3.10.0-957.10.1.el7.x86_64 (IZwz97TBgo9lkabnat2LO8z) 06/02/2019_x86_64_ (2 CPU) 06:48:30 PM CPU %user % Nice % System % IOwait % Steal % Idle 06:48:35 PM all 0.10 0.00 0.10 0.00 0.00 99.80 06:48:30 PM kbMEMfree Kbmemused %memused kbbuffers kbcached kbcommit %commit kbactive kbinact kbdirty 06:48:35 PM 1772012 2108392 54.33 102816 1634528 784940 20.23 793328 1164704 0Copy the code

-A– Displays the health of all devices (CPU, memory, disk).
-u– Displays the load of all cpus.
-d– Displays the usage of all disks.
-r– Displays memory usage.
-n– Displays the network running status.

Viewing memory usage – free

[root ~]# free
              total        used        free      shared  buff/cache   available
Mem:        1016168      323924      190452         356      501792      531800
Swap:             0           0           0
Copy the code

Virtual memory statistics – vmstat.

[root ~]# vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 2  0      0 204020  79036 667532    0    0     5    18  101   58  1  0 99  0  0
Copy the code

CPU statistics – mpstat.

[root ~]# mpstat Linux 3.10.0-957.5.1.el7.x86_64 (iZ8vba0s66jjlfmo601w4xZ) 05/30/2019 _x86_64_ (1 CPU) 01:51:54am CPU %usr %nice % SYS % IOwait %irq % Soft % Steal %guest % gnICE % IDLE 01:51:54 AM ALL 0.71 0.00 0.17 0.04 0.00 0.00 0.00 0.00 0.00 99.07Copy the code

View process memory usage – pmap.

[root ~]# ps PID TTY TIME CMD 4581 pts/0 00:00:00 bash 5664 pts/0 00:00:00 ps [root ~]# pmap 4581 4581: -bash 0000000000400000 884K r-x-- bash 00000000006dc000 4K r---- bash 00000000006dd000 36K rw--- bash 00000000006e6000 24K rw-- [anon] 0000000001de0000 400K rw-- [anon] 00007F82FE805000 48K r-x-- libnSS_FILes-2.17.so 00007f82FE811000 2044 k -- -- -- -- -- libnss_files - 2.17. So...Copy the code

Report device CPU and I/O statistics – iostat.

[root ~]# iostat Linux 3.10.0-693.11.11.el7.x86_64 (iZwz97tbgo9lkabnat2lo8Z) 06/26/2018 _x86_64_ (1 CPU) avg-CPU: %user %nice % System % IOwait % Steal %idle 0.79 0.00 0.20 0.04 0.00 98.97 Device: TPS kB_read/s kB_wrtn/s kB_read kB_wrtn VDA 0.85 6.78 21.32 2106565 6623024 VDB 0.00 0.01 0.00 2088 0Copy the code

Display all PCI devices – lSPCI.

[root ~]# lSPCI 00:00.0 Host bridge: Intel Corporation 440FX-82441FX PMC [Natoma] (Rev 02) 00:01.0 ISA Bridge: Intel Corporation 440FX-82441FX Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 00:01.1 IDE Interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] 00:01.2 USB Controller: Intel Corporation 82371SB PIIX3 USB [Natoma/Triton II] (Rev 01) 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (Rev 03) 00:02.0 VGA Compatible Controller: Cirrus Logic GD 5446 00:03.0 Ethernet Controller: Red Hat, Inc. Virtio Network Device 00:04.0 Communication Controller: Red Hat, Inc. Virtio Console 00:05.0 SCSI Storage Controller: Red Hat, Inc. Virtio Block Device 00:06.0 SCSI Storage Controller: Red Hat, Inc. Virtio Block Device 00:07.0 Unclassified Device [00FF]: Red Hat, Inc. Virtio Memory BalloonCopy the code

Displays the status of the interprocess communication facility – IPCS.

[root ~]# ipcs

------ Message Queues --------
key        msqid      owner      perms      used-bytes   messages    

------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status      

------ Semaphore Arrays --------
key        semid      owner      perms      nsems
Copy the code

Shell programming

As mentioned before, Shell is an application program that connects the user and the operating system. It provides a human-computer interaction interface (interface) through which the user accesses the services of the operating system kernel. Shell script is a script program written for the Shell, we can use the Shell script for system management, but also through it for file operations. In short, Shell scripting should be a standard skill for anyone using Linux.

There is a lot of knowledge about Shell scripts on the Internet. I am not going to give a comprehensive and systematic explanation of Shell scripts here. Let’s get a perceptual understanding of Shell scripts through the following code.

Example 1: Enter two integers m and n and compute the sum of integers from m to n.

#! /usr/bin/bashPrintf'm = 'read m printf 'n =' read n a=$m sum=0 while [$a -le $n] do sum=$[sum + a] a=$[a + 1] done echo '  '$sumCopy the code

Example 2: Automatically create folders and a specified number of files.

#! /usr/bin/bashPrintf 'Input file name:' read file printf 'Input file number (<1000): Else if [-e $dir -a -d $dir] then RM -rf $dir else if [-e $dir -a -d $dir $dir -a -f $dir ] then rm -f $dir fi fi mkdir -p $dir index=1 while [ $index -le $num ] do if [ $index -lt 10 ] then pre='00' elif [ $index -lt 100 ] then pre='0' else pre='' fi touch $dir'/'$file'_'$pre$index index=$[ index + 1 ] done fiCopy the code

Example 3: Automatically install the specified version of Redis.

#! /usr/bin/bashinstall_redis() { if ! which redis-server > /dev/null then cd /root wget $1$2'.tar.gz' >> install.log gunzip /root/$2'.tar.gz' tar -xf /root/$2'.tar' CD /root/$2 make >> install.log make install >> install.log echo 'install' else echo 'install' install_redis 'http://download.redis.io/releases/' $1Copy the code

The related resources

Common shortcut keys on the Linux COMMAND line

shortcuts	Functional specifications
tab	Automatic completion command or path
Ctrl+a	Move the cursor to the beginning of the command line
Ctrl+e	Move the cursor to the end of the command line
Ctrl+f	Move the cursor one character to the right
Ctrl+b	Move the cursor one character to the left
Ctrl+k	Clipping characters from the cursor to the end of the line
Ctrl+u	Cut the character from the cursor to the beginning of the line
Ctrl+w	Cut a word in front of the cursor
Ctrl+y	Copy the clip to name the content of the clip
Ctrl+c	Interrupts a task in progress
Ctrl+h	Deletes a character before the cursor
Ctrl+d	Exit the current command line
Ctrl+r	Search history command
Ctrl+g	Exit the history command search
Ctrl+l	Clear everything on the screen and open a new line at the top of the screen
Ctrl+s	Lock the terminal so that it is temporarily unable to enter content
Ctrl+q	Exiting terminal Lock
Ctrl+z	Stops the task being executed on the terminal and puts it in the background
!!!!!	Run the previous command
! digital	Run the history command corresponding to the number
! The letter	Execute the most recent alphabetic command
! $ / Esc+.	Gets the last argument to the previous command
Esc+b	Move to the beginning of the current word
Esc+f	Move to the end of the current word

Man Refer to the content description of the command manual

The title in the manual	Functional specifications
NAME	Description and introduction of commands
SYNOPSIS	Use the basic syntax of the command
DESCRIPTION	Use the detailed description of the command, the function of each parameter, sometimes such information will appear in OPTIONS
OPTIONS	This section describes command parameters
EXAMPLES	Example of using this command
EXIT STATUS	Exit status code at the end of a command. 0 indicates that the command is successfully executed
SEE ALSO	Other commands or information related to a command
BUGS	Description of defects associated with the command
AUTHOR	Introduction to the author of the command

Go to the original link

Play with the Linux operating system