Common operations and concepts

shortcuts

  • Tab: command and file name completion.
  • Ctrl+C: Interrupts a running program;
  • Ctrl+D: End Of File, EOF

For help

–help

Instruction basic usage and option introduction.

man

Man is short for manual, which displays detailed information about instructions. When man date is executed, date (1) appears, where the number represents the instruction type. Commonly used numbers and their types are as follows:

Code name type
1 Instructions or executable files that the user can manipulate in the shell environment
5 The configuration file
8 Management commands that system administrators can use

Linux Configuration Man Manual.

# 1. Download the source package$ wget https://src.fedoraproject.org/repo/pkgs/man-pages-zh-CN/manpages-zh-1.5.1.tar.gz/13275fd039de8788b15151c896150bc4/manpag Es - useful - 1.5.1. Tar. Gz# 2. Unpack and install$tar -zxvf manpages-zh-1.5.1.tar.gz $./configure --disable-zhtw --prefix=/usr/local/zhman
$ make && make install

# 3. Create cman command as Chinese query
$ cd ~
$ echo "alias cman='man -M /usr/local/zhman/share/man/zh_CN' " >> .bash_profile
Copy the code

info

Info is similar to MAN, but info breaks the document into pages, and each page can be jumped to.

doc

/usr/share/doc contains a complete set of documentation files for the software.

To turn it off

who

Before shutting down the device, run the who command to check whether other users are online.

sync

To speed up the read/write speed of disk files, file data in the memory is not immediately synchronized to the disk. Therefore, sync must be performed before the shutdown.

shutdown

$shutdown [-krhc] time [information] -k: indicates that the system will not be shutdown. -r: indicates that the system will be restarted after the shutdown. -h: indicates that the system will be shutdown immediately after the shutdownCopy the code

Package management tool

RPM and DPKG are the two most common types of package management tools:

  • RPM, known as Redhat Package Manager, was first developed and implemented by Red Hat, and then accepted by GNU open source operating system and became the established software standard of many Linux systems. YUM is based on RPM and has dependency management and software upgrade functions.
  • Competing with RPM is the Debian operating system-based DEB Package management tool DPKG, called Debian Package, which is similar in functionality to RPM.

release

A Linux distribution is an integrated version of the Linux kernel and various applications.

Package management tool based Commercial release Community distribution
RPM Red Hat Fedora / CentOS
DPKG Ubuntu Debian

VIM three modes

  • Command mode: VIM’s default mode, which can be used to move the cursor to view content.
  • Insert mode: press the “I” button to enter and edit the text;
  • Bottom-line mode: Enter after pressing the “:” button, which is used for saving and exiting operations.

disk

Name of the disk

Every piece of hardware in Linux is treated as a file, including the disk. The disk name is based on the disk interface type. Common disk names are as follows:

  • IDE disk: /dev/hd[a-d]
  • SATA/SCSI/SAS disk: /dev/sd[a-p]

Viewing Disk Space

View the disk space usage:

df -h

-h: human abbreviation to display the result in readable form (i.e. with units such as M/G, if this parameter is not added, the number displayed is in units of B)

df -h
File system capacity used Available Used % mount point
# devtmpfs 1.2g 0 1.2g 0% /dev
/dev/shm
TMPFS 1.9g 292K 1.9g 1% /run
TMPFS 1.9g 0 1.9g 0% /sys/fs/cgroup
# /dev/vda1 40G 8.0g 30G 22% /
Copy the code

Check the size of the current directory:

du -sh

  • -h User-friendly display
  • -s Indicates the recursive size of the entire directory
$ du -sh
# 429M	.
Copy the code

partition

The partition table

There are two main formats for disk partition tables.

  • MBR partition table (more limited)
  • GPT partition table (newer and less restrictive)

MBR

In the MBR, the first sector is the most important. It contains the Master Boot Record (MBR) and partition table (Partition table). The main boot record and partition table account for 446 bytes and 64 bytes respectively.

Linux also treats partitions as files. Partition files are named as disk file name + number, for example, /dev/sda1. Note that the logical partition numbers start at 5.

GPT

A sector is the smallest unit of storage for a disk. Older disks typically had a sector size of 512 bytes, while newer disks support 4 K.

Boot detection program

BIOS

BIOS (Basic Input/Output System) is a firmware (software embedded in hardware). BIOS programs are stored in read-only memory that is not lost after a power failure. GPT partition tables cannot be read.

To install multiple booting, it is best to install Windows before Linux. Because Windows overwrites the main Boot Record (MBR) when installing, Linux can choose to install the boot manager in the boot sector of the main boot Record (MBR) or another partition, and can set the boot manager menu.

UEFI

BIOS cannot read the GPT partition table while UEFI can.

The file system

composition

The main components are as follows:

  • Inode: A file occupies one inode, records the attributes of the file, and records the block id of the file.
  • Block: Records the contents of a file. If a file is too large, multiple blocks will be occupied.

Among other things:

  • Superblock: Records the overall information about a file system, including the total number of inodes and blocks, usage, and remaining amount, as well as the format and related information of the file system.
  • Block bitmap: a bitmap that records whether a block is used.


Directory configuration

To keep directory structures consistent across Linux distributions, the Filesystem Hierarchy Standard (FHS) specifies the Linux directory structure. The basic three directories are as follows:

  • /(root, root directory)
  • /usr(Unix Software Resource) : All system default software is installed in this directory;
  • /varVariable: stores data files during system or program running.


Basic operations on files and directories

ls

Lists information about a file or directory. The directory information is the file contained therein.

$ls [- aAdfFhilnrRSt] file | dir - a: list all the files - d: only list the directory itself - l: by long serial data list, include files of attribute data and authority and so onCopy the code

cd

Replace the current directory.

$ cd[Relative path or absolute path]Copy the code

mkdir

Create a directory.

$mkdir [-mp] Directory name -m: sets directory permission -p: creates a directory recursivelyCopy the code

rmdir

To delete a directory, the directory must be empty.

$rmdir [-p] Directory name -p: deletes a directory recursivelyCopy the code

touch

Update file time or create a new file.

$touch [-acdmt] filename -a: updates the ctime. Atime -c: updates the ctime. If the file does not exist, no new file will be created"Date or time"-t: Indicates the update time instead of the current time. The format is [YYYYMMDDhhmm].Copy the code

cp

Copy files. If there are more than two source files, the destination file must be a directory.

$ cp [-adfilprsu] sourceDestination-a: equivalent to -dr --preserve=all -d: If the source file is a linked file, copy the attributes of the linked file instead of the file itself -i: If the target file already exists, it will be asked before overwriting. -p: Copy the file along with the attributes of the file. -r: Recursive replication -u: destination ratiosource-- Preserve =all: add SELinux attributes, links, xattr, etcCopy the code

rm

Delete files.

$rm [-fir] File or directory -r: recursive deletionCopy the code

mv

Move files.

$ mv [-fiu] sourcedestination $ mv [options] source1 source2 source3 .... Directory -f: force Specifies whether to overwrite a file if the target file already existsCopy the code

Modify the permissions

A group of permissions can be represented by numbers. In this case, the three bits of a group of permissions are regarded as binary digits. The weight of each bit from left to right is 4, 2, and 1, that is, the numeric weight of each permission is R: 4, W: 2, and x: 1.

Changing read/write Permissions

$ chmod [-R] xyz dirname/filename
Copy the code

Example: Change the permission of the.bashrc file to -rwxr-xr–.

$ chmod 754 .bashrc
Copy the code

You can also use symbols to set permissions.

$chmod [ugoa] [+-=] [RWX] dirname/filename -u: owner -g: owning group -o: others -a: all - + : add permission - - : remove permission -= : set permissionCopy the code

Example: Add write permission to all users of the.bashrc file.

$ chmod a+w .bashrc
Copy the code

Change the owner of a file or directory

$ chown username dirOrFile
Copy the code

Use the -r option to recursively change the owner of all files under this table:

$ chown -R weber server/
Copy the code

The default permissions

  • Default file permission: The file has no executable permission by default, so666, that is,-rw-rw-rw- 。
  • Default directory permissions: The directory must be accessible, that is, it must have executable permissions777, that is,drwxrwxrwx.

link

Linux ln command


$ln [-sf] source_filename dist_filename -s: indicates an entity link by default. -s indicates a symbolic link. -f: deletes the target file if it existsCopy the code

Physical link

Creates an entry in the directory with the file name and inode number. The inode is the inode of the source file.

Delete any entry and the file still exists, as long as the number of references is not zero.

There are the following restrictions: you cannot link directories across file systems.

$ ln /etc/crontab .
$ ll -i /etc/crontab crontab
# 34474855 -rw-r--r--. 2 root root 451 Jun 10 2014 crontab
# 34474855 -rw-r--r--. 2 root root 451 Jun 10 2014 /etc/crontab
Copy the code

A symbolic link

Symbolic link file stores the absolute path of the source file. When reading the file, it will locate the source file. It can be understood as a Shortcut of Windows.

When the source file is deleted, the link file cannot be opened.

Because you are recording paths, you can create symbolic links for directories.

$ ll -i /etc/crontab /root/crontab2
# 34474855 -rw-r--r--. 2 root root 451 Jun 10 2014 /etc/crontab
# 53745909 lrwxrwxrwx. 1 root root 12 Jun 23 22:31 /root/crontab2 -> /etc/crontab
Copy the code

Obtaining file contents

cat

Get file contents.

$cat [-abentv] filename -n: prints the travel id, along with the blank line number, -b does notCopy the code

tac

Is the reverse of cat, printing from the last line.

more

Unlike CAT, it can view the contents of a file page by page, which is suitable for viewing large files.

less

Similar to More, but with the ability to scroll forward.

head

Get the first few lines of the file.

$head [-n number] filename -n: indicates the number of lines to displayCopy the code

tail

It’s the reverse of head, but it gets the last few lines.

od

Displays binary files in character or hexadecimal format.

Command and file search

which

Command search. The which directive looks for eligible files in the directory set by the environment variable $PATH.

$ which [-a] command-a: Lists all commands, not only the first oneCopy the code

whereis

File search. The whereis command first strips the prefix space and prefix in filename. At the beginning of any character, and then in the database (/ var/lib/slocate slocate. Db) find the match the filename after the above processing of binary file, source files, and help manual.

You can manually update the database before using updatedb
$ whereis [-bmsu] dirname/filename
Copy the code

locate

File search. You can search by keyword or regular expression (remembering only the partial name of a file).

Locate uses the /var/lib/mlocate/ database for searching. It is stored in memory and updated daily, so you can’t use Locate to search for new files.

You can manually update the database before using updatedb$locate [-ir] keyword -r: indicates the regular expressionCopy the code

find

File search. You can search using file properties and permissions.

The find command traverses the current working directory and its subdirectories, which consumes hard disk resources and is less efficient than whereis and locate.

$ find [basedir] [option]
# example: find . -name "shadow*"
Copy the code

(1) Time

-mtime +n: lists the files whose content has been modified before n days. -mtime -n: lists the files whose content has been modified before n days. Newer file: Lists files that are updated within N days. - Newer file: Lists files that are newer than fileCopy the code

The time ranges for +4, 4, and -4 instructions are as follows:


② Options related to the file owner and group

-uid n-gid n -user name -group name -nouser: searches for files whose owner does not exist in /etc/passwd. -nogroup: searches for files whose owning group does not exist in /etc/groupCopy the code

③ Options related to file permissions and names

-name filename-size [+-] size: Searches for files larger (+) or smaller (-) than size. The SIZE ranges from c to byte and k to 1024bytes. So, to find a file larger than 50KB, it's -size + 50K -typeType-perm mode: searches for files with the same permission as mode. -perm-mode: searches for files with the same permission as mode. -perm /mode: searches for files with any modeCopy the code

Compression and packaging

Compressed file name

There are many compressed file names under Linux, common ones are as follows:

extension Compression programs
*.Z compress
*.zip zip
*.gz gzip
*.bz2 bzip2
*.xz xz
*.tar The tar program packages data without compression
*.tar.gz Tar program packaged file, gzip compressed
*.tar.bz2 Tar package files, compressed by bzip2
*.tar.xz Tar program package file, after Xz compression

Compressed instruction

gzip

Gzip is the most widely used compression command in Linux, and can unlock files compressed by COMPRESS, zip, and gzip.

After gzip compression, the source file does not exist.

There are 9 different compression levels available.

You can use Zcat, Zmore, zless to read the contents of compressed files.

$ gzip [-cdtv#] filename-c: outputs compressed data to the screen. -d: decompresses the file. -t: checks whether errors occur in the compressed file# : # indicates the compression level. The larger the number, the higher the compression ratio. The default value is 6
Copy the code

bzip2

Provides a higher compression ratio than GZIP.

View the commands bzcat, bzmore, bzless, and bzgrep.

$ bzip2 [-cdkzv#] filename-k: Saves the source fileCopy the code

xz

Provides a better compression ratio than Bzip2.

As you can see, the compression ratios of Gzip, Bzip2 and Xz are constantly optimized. Note, however, that the higher the compression ratio, the longer the compression time.

Run xzcat, xzmore, xzless, and xzgrep.

$ xz [-dtlkc#] filename
Copy the code

packaging

The compress command can compress only one file, while packaging can pack multiple files into one large file. Tar can be used not only for packaging, but also for compression of packaged files using gzip, bzip2, xz.

$tar [z | - | j - j] [CV] the new tar file] [-f filename... = = compress $tar [z | - j | - j] [TV] [-f existing tar file] = = see $tar [z | - j | - j] [xv] [-f existing tar file] [-] C directory = = unzip - z: use zip; -j: use bzip2; -j: xz is used. -c: Creates a package file. -t: view the files in the package file. -x: unpack or decompress. -v: displays the file name being processed during compression or decompression. -f: filename: indicates the file to be processed. -c directory: Decompresses files in a specific directory.Copy the code
use The command
Packing compression Tar -jcv -f filename.tar.bz2 Specifies the name of the file or directory to be compressed
Check to see tar -jtv -f filename.tar.bz2
unzip Tar -jxv -f filename.tar.bz2 -c Specifies the directory to be decompressed

Bash

Bash is one type of Shell where you can request services from the kernel.

features

  • Command history: Records used commands
  • Command and file completion: Shortcut key: TAB
  • Name alias: For examplell 是 ls -alThe alias
  • shell scripts
  • Wildcard: for examplels -l /usr/bin/X\*list/usr/binAll of the following files begin with X

Variable operating

To assign a variable, use =.

${}; ${};

Output variables using the echo command.

$ x=abc
$ echo $x
$ echo ${x}
Copy the code

Variable content that contains Spaces must be quoted in double or single quotes.

  • The special characters inside the double quotes can retain the original features. For example, if x=”lang is $lang “, the value of x is lang is zh_tw.utF-8.
  • Special characters in single quotes are special characters themselves. For example, x=’lang is $lang ‘, the value of x is lang is $lang.

The result of an instruction can be assigned to a variable using either an ‘instruction’ or $(instruction). For example, version=$(uname -r), the value of version is 4.15.0-22-generic.

You can use the export command to convert custom variables into environment variables that can be used in a subroutine, which is a child Bash generated from the current Bash.

Bash variables can be declared as arrays and integer numbers. Note that numeric types do not have floating point numbers. If not declared, the default is a string. Declare variables using the declare command:

$ declare[-aixr] variable -a: the array type -i: the integer type -x: the environment variable -r: the array typereadonlytypeCopy the code

Use [] to index an array:

$ array[1]=a
$ array[2]=b
$ echo ${array[1]}
Copy the code

Command search order

  • Execute instructions in absolute or relative paths, such as /bin/ls or./ls;
  • The instruction is executed by finding the alias;
  • Executed by Bash’s built-in instructions;
  • Find the first instruction to execute in the order of the search PATH specified by the $PATH variable.

Data flow redirection

Redirection refers to the use of files instead of standard input, standard output, and standard error output.

1 code The operator
Standard input (STDIN) 0 Or < < <
Standard output (STDOUT) 1 > or > >
Standard error output (STDERR) 2 > 2 or more > >

< and > indicate redirection in override mode. The representations of >> and << are redirected append.

/dev/null is thrown into the trash.

If you want to redirect both stdout and stDOut to a file, you need to convert one output to another, for example, 2>&1 to convert stDout to stDout.

Bashrc (standard input and error output) write and overwrite to list
$ find /home -name .bashrc > list 2>&1 
Append (standard input and error output) the path to the file named ".bash" in /home
$ find /home -name ".bash*" >> list 2>&1
Copy the code

Pipeline instruction

Take the standard output of one command as the standard input of another.

$ ls -al /etc | less
Copy the code

Extract the instruction

cut

You slice the data, you pull out what you want.

The sharding process happens line by line.

$cut -d: delimiter -f: After the -d delimiter, run -f n to fetch the NTH interval. -C: To fetch the interval in charactersCopy the code

Example 1: Last displays the login information and retrieves the user name.

$ last
# root PTS /1 192.168.201.101 Sat Feb 7 12:35 Still logged in
# root PTS /1 192.168.201.101 Fri Feb 6 12:13-18:46 (06:33)
# root PTS /1 192.168.201.254 Thu Feb 5 22:37-23:53 (01:16)

$ last | cut -d ' ' -f 1
Copy the code

Example 2: Take the output of export and extract all strings after the 12th character.

$ export
# declare -x HISTCONTROL="ignoredups"
# declare -x HISTSIZE="1000"
# declare -x HOME="/home/dmtsai"
# declare -x HOSTNAME="study.centos.vbird"
#... .....

$ export | cut -c 12-
Copy the code

Ordering instructions

sort

Used for sorting.

$sort [-fbmnrtuk] [file or stdin] -f: ignore case -b: ignore the uppermost space -m: sort by the name of the month, such as JAN, DEC -n: use numbers -r: reverse sort -u: Equivalent to unique, repeated content occurs only once. -t: delimiter, default to TAB -k: specifies the sort intervalCopy the code

Example: /etc/passwd File contents are separated by:, requiring sorting in the third column.

$ cat /etc/passwd | sort -t ':' -k 3
# root:x:0:0:root:/root:/bin/bash
# dmtsai:x:1000:1000:dmtsai:/home/dmtsai:/bin/bash
# alex:x:1001:1002::/home/alex:/bin/bash
# arod:x:1002:1003::/home/arod:/bin/bash
Copy the code

uniq

You can take only one duplicate data.

$uniq [-ic] -i: ignores case and c: countsCopy the code

Example: Get the total number of logins per person

$ last | cut -d ' ' -f 1 | sort | uniq -c
# 1
# 6 (unknown
# 47 dmtsai
# 4 reboot
# 7 root
# 1 wtmp
Copy the code

Two-way output redirection

Output redirection redirects the output to a file, and TEE not only does this, but also preserves the output on the screen. That is, with the TEE directive, an output is sent to both the file and the screen.

$ tee [-a] file
Copy the code

Character conversion instruction

tr

Used to delete or replace characters in a line.

$ tr [-ds] SET1 ... -d: Delete the string SET1 from the lineCopy the code

For example, convert all lowercase information output from last to uppercase.

$ last | tr '[a-z]' '[A-Z]'
Copy the code

col

Converts a TAB character to a space character.

$col [-xb] -x: converts the TAB key to an equal space keyCopy the code

expand

Convert TAB to a certain number of Spaces, 8 by default.

$expand [-t] file -t: Indicates the number of Spaces converted from TABCopy the code

join

Merge the row that has the same data.

$join [-ti12] file1 file2 -t: delimiter. The default value is space. -I: ignores case differencesCopy the code

paste

Paste the two lines directly together.

$paste [-d] file1 file2 -d: delimiter. The default value is TABCopy the code

Partition instruction

split

Divide a file into multiple files.

$split [-bl] file PREFIX -b: splits the file by size, including b, k, m, and so on. -l: splits the file by line. - PREFIX: indicates the leading name of the partition fileCopy the code

Regular expression

grep

G/RE/P (globally search a regular expression and print), which uses regular expressions to search and print globally.

$grep [-acinv] [--color=auto] Search string filename -c: counts the number of matched rows. -I: ignores case. -n: displays the line number. Reverse selection, that is, to show the line without the contents of the searched string --color=auto: the keyword found is coloredCopy the code

Example: Extract the line containing the string (note that the default is –color=auto, so the string is colored in Linux)

$ grep -n 'the' regular_express.txt
8:I can't finish the test.
12:the symbol '*' is represented as start.
15:You are the best is mean you are the no. 1.
16:The world Happy is the same with "glad".
18:google is the best tools for search keyword
Copy the code

Example: The regular expression a{m,n} is used to match the character a m~n times. {and} need to be escaped because they have special meaning in the shell.

$ grep -n 'a \ {2, 5 \}' regular_express.txt
Copy the code

printf

Used to format output. It is not a pipe command and requires the $() form when passing data to printf.

$ printf '%10s %5i %5i %5i %8.2f \n' $(cat printf.txt)
    DmTsai    80    60    92    77.33
     VBird    75    55    80    70.00
       Ken    60    90    70    73.33
Copy the code

awk

AWK is a language for processing text files. It is a powerful text analysis tool created by Alfred Aho, Peter Weinberger, and Brian Kernighan. AWK is the initials of the three founders.

Process management

Check the process

ps

View information about processes at a point in time.

Example: View your own process

$ ps -l
Copy the code

Example: View all system processes

$ ps aux
Copy the code

Example: View specific processes ([pipe](# 8, pipe instructions), re)

$ ps aux | grep threadx
Copy the code

Example Query information about running processes

$ ps -ef
Copy the code

Eg: Query the process that belongs to user colin115

$ ps -ef | grep colin115
$ ps -lu colin115
Copy the code

pstree

View the process tree.

Example: View the tree of all processes

$ pstree -A
Copy the code

top

Displays process information in real time.

Example: Refresh every two seconds

$ top -d 2
Copy the code

netstat

View the process that occupies the port

Example: View processes for a particular port

$ netstat -anp | grep port
Copy the code

lsof

Check the status of processes occupied by ports:

$ lsof -i:3306
Copy the code

View the files opened by user username’s process

$ lsof -u username
Copy the code

Query the currently open file of the init process

$ lsof -c init
Copy the code

Query open files for the specified process ID(23295) :

$ lsof -p 23295
Copy the code

pgrep

Query process ID (suitable for remembering only partial process fields)

$pgrep Find process# eg: Query the process whose process name contains re
$ pgrep -l re
# 2 kthreadd
# 28 ecryptfs-kthrea
# 29515 redis-server
Copy the code

Terminate the process

Kill the Process with the specified PID (PID is the Process ID)

$ kill PID
Copy the code

Kill related processes

$ kill3434-9Copy the code

Kill job number (job number)

$ kill %job
Copy the code

The integrated use of

Terminate all processes under user colin115 whose process names start with av_ :

$ ps -u colin115 |  awk '/av_/ {print "kill -9 " $1}' | sh
Copy the code

Alter table colin115 alter table colin115 alter table colin115 alter table colin115

$ ps -fe| grep colin115|grep HOST |awk '{print $2}' | xargs kill- 9;Copy the code

Process status

Process status reflects changes in process execution. These states change as the process executes and external conditions change. In the three-state model, the process state is divided into three basic states, namely running state, ready state and blocked state.

state instructions
R running or runnable (on run queue)

Executing or executable, the process is in the execution queue.
D uninterruptible sleep (usually I/O)

Uninterruptible sleep state, can process signal, with delay. Non-interruptible blocking, usually I/O blocking.
S interruptible sleep (waiting for an event to complete)

Interruptible blocking when a process is waiting for an event to complete.
Z zombie (terminated but not reaped by its parent)

Dead, the process has terminated but has not yet been retrieved by its parent.
T stopped (either by a job control signal or because it is being traced)

Termination, the process can either be terminated by a job control signal or is being tracked.


**R(TASK_RUNNING) Indicates the executable status **

Only processes in this state can run on the CPU. Multiple processes may be in an executable state at the same time, and their task_struct structures (process control blocks) are placed in the corresponding CPU’s executable queue (a process can only appear in one CPU’s executable queue). The task of the process scheduler is to select a process from each CPU’s executable queue to run on that CPU.

Many operating system textbooks define RUNNING processes on the CPU and READY processes that are RUNNING but not yet scheduled to execute. These two states are unified under Linux as TASK_RUNNING.

S(TASK_UNINTERRUPTIBLE) Shallow sleep state

A process in this state is suspended because it is waiting for something to happen, such as a socket connection or a semaphore. The task_struct structure of these processes is put into the wait queue for the corresponding event. When these events occur (triggered by an external interrupt, or by another process), one or more processes in the corresponding wait queue wake up.

Using the ps command, we can see that, in general, most processes in the process list are in TASK_INTERRUPTIBLE state (unless the machine is heavily loaded). After all, the CPU is only one or two processes, often dozens or hundreds of processes, if not the vast majority of processes are asleep, how can the CPU respond.

D(TASK_UNINTERRUPTIBLE) Deep sleep (a state of uninterruptible sleep)

This occurs when you are waiting for a critical resource or waiting for a system call to end.

Similar to the TASK_INTERRUPTIBLE state, the process is asleep, but is not interruptible at this point. Uninterruptible does not mean that the CPU does not respond to interrupts from external hardware, but that the process does not respond to asynchronous signals. In most cases, processes should always respond to asynchronous signals when they are asleep. Otherwise you'll be surprised to find that 'kill -9' can't kill a sleeping process! It is easy to understand why the ps command almost never sees a TASK_UNINTERRUPTIBLE state, but always a TASK_INTERRUPTIBLE state.Copy the code

T(Stopped) The Stopped state

A process pauses to accept some processing. If the process being debugged is in this state, Linux uses the TASK_STOPPED macro to indicate this state. The process enters the pause state when it receives SIGSTOP, SIGTSTP, SIGTTIN, or SIGTTOU. It can be sent a SIGCONT signal to convert the process to a runnable state. The process enters this state when it receives any signals during debugging. When Ctrl+ Z is pressed, the system kernel sends SIGSTOP to the front-end process. Similarly, the command bg/fg causes bash to send SIGCONT to the process.Copy the code

In fact, sending a signal directly at the terminal causes the process to react in the same way

$ kill -STOP PCB_NUM
$ kill -CONT PCB_NUM
Copy the code

Z (task_dead-exit_zombie) Zombie status

  • ** Orphan processes: ** If a parent exits while one or more of its children are still running, those children become orphan processes. Orphan processes are adopted by the init process (process number 1), which collects state for them.

  • ** Zombie process: ** A process creates a child process using fork. If the child exits and the parent does not call wait or WaitPID to get the child’s state, the child’s process descriptor remains in the system. This process is called rigor mortis.

Hazard:

The system provides a mechanism to ensure that whenever the parent wants to know the status of the child process, it can get it. When each process exits, the kernel frees all of its resources, including open files and occupied memory. However, certain information is still reserved for it (including the process ID, the termination status of the process, the amount of CPU time taken by the The process, etc.). It is not released until the parent retrieves it via wait/waitPID. But this is causing the problem, if the process without calling wait/waitpid, retain the information should be released, the process will have been occupied, but the system can use process number is limited, if a zombie, the product of the would, because there is no available process and cause system can't generate new process. This is a zombie process hazard and should be avoided.Copy the code

SIGCHLD

When a child process changes its state (stops running, continues running, or exits), two things happen to the parent process:

  • SIGCHLD signal is obtained.
  • The waitpid() or wait() call returns.

The SIGCHLD signal sent by the child process contains information about the child process, such as process ID, process status, and CPU usage time.

When a child exits, its process descriptor is not released immediately so that the parent process can get information about the child. The parent process uses wait() and waitpid() to get information about an exited child.

wait()

pid_t wait(int *status)
Copy the code

The parent process calls wait() and blocks until it receives a SIGCHLD signal that a child process has quit, after which wait() destroys the child process and returns.

If successful, return the process ID of the collected child process; If the calling process has no children, the call fails and returns -1 with errno set to ECHILD.

The status parameter is used to store the status of the collected child process when it exits. You can set this parameter to NULL if you don’t care how the child process died and just want to kill it.

waitpid()

pid_t waitpid(pid_t pid, int *status, int options)
Copy the code

Wait () is the same as wait(), but with two more user-controlled parameters pid and options.

The PID parameter indicates the ID of a child process, indicating that only the SIGCHLD signal for the child process to exit is concerned. If pid=-1, wait() is the SIGCHLD signal for all child processes to exit.

Options are WNOHANG and WUNTRACED. WNOHANG makes the waitpid() call non-blocking, meaning that it will return immediately and the parent can continue to perform other tasks.

An orphan

If a parent exits while one or more of its children are still running, they become orphan processes.

Orphan processes are adopted by the init process (process number 1), which collects state for them.

Since orphaned processes are adopted by the init process, orphaned processes do no harm to the system.

zombies

The process descriptor for a child process is not released when the child exits, but only when the parent process obtains the child’s information via wait() or waitpid(). If the child exits without the parent calling wait() or waitpid(), the child’s process descriptor remains in the system and is called a zombie process.

The ps command displays the zombie process in Z (zombie) state.

The number of process ids available to the system is limited. If a large number of zombie processes are generated, the system cannot generate new processes because there are no available process ids.

To kill a large number of zombie processes in the system, you simply kill the parent, and the zombie process becomes an orphan and is adopted by init, which releases all resources occupied by the zombie process and terminates the zombie process.

Performance monitoring

Monitor the CPU

Viewing CPU Usage

$ sar -u

# eg:
$ sar -u 1 2
[/home/weber#]sar -u 1 2Linux 2.6.35-22-generic-pae (MyVPS) 06/28/2014 _i686_ (1 CPU) 09:03:59 AM CPU %user %nice %system %iowait % Steal %idle 09:04:00 AM all 0.00 0.00 0.50 0.00 99.50 09:04:01 AM all 0.00 0.00 0.00 100.00Copy the code

The following two parameters indicate the frequency of monitoring, such as 1 and 2 in the example, which means sampling once per second, two times in total.

View the AVERAGE CPU load

$ sar -q 1 2
Copy the code

After SAR is specified with -q, you can view the number of processes in the running queue, process size on the system, and load average.

Query memory

Viewing memory Usage After -r is specified in SAR, the memory usage can be viewed.

$ sar -r 1 2
# 09:08:48 AM kbmemfree kbmemused %memused kbbuffers kbcached kbcommit %commit kbactive kbinact
# 09:08:49 AM 17888 359784 95.26 37796 73272 507004 65.42 137400 150764
# 09:08:50 AM 17888 359784 95.26 37796 73272 507004 65.42 137400 150764
# Average: 17888 359784 95.26 37796 73272 507004 65.42 137400 150764
Copy the code

Viewing Memory Usage

$ free -m
Copy the code

Query page exchange

Viewing the status of page swapping When pages are swapping, the throughput of the server decreases greatly. When the server is in bad condition, if you suspect that page swapping is occurring due to insufficient memory, you can use the SAR -w command to confirm that a large number of swapping is occurring.

$ sar -W 1 3
Copy the code

Querying disk Usage

View the disk space usage

$ df -h
Copy the code

Example Query the space usage of the current directory

$du -sh -h is the humanized display s is the recursive size of the entire directoryCopy the code

View the sorted size of all folders in this directory

$ for i in `ls`; do du -sh $i; done | sort
# or
$ du -sh `ls`
Copy the code

Web tools

Example Query network services and ports

The netstat command is used to display various network-related information, such as network connections, routing tables, Interface Statistics, Masquerade connections, and Multicast Memberships.

List all ports (listening and unlistening):

$ netstat -a
Copy the code

List all TCP ports:

$ netstat -at
Copy the code

List the status of all monitored services:

$ netstat -l
Copy the code

Use netstat to query ports:

$ netstat -antp | grep 6379
# TCP 0 0 127.0.0.1:6379 0.0.0.0:* LISTEN 25501/redis-server

$ ps 25501
# PID TTY STAT TIME COMMAND
# 25501? Ssl 28:21 ./redis-server ./redis.conf
Copy the code

Eg. Query port 7902 now running what procedures

Query the PID of the process using this port;
$ lsof -i:7902
# COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
# WSL 30294 TUapp 4U IPv4 447684086 TCP 10.6.50.37: tnoS-dp (LISTEN)

Select * from process 30294;
$ ps -fe | grep 30294
# tdev5 30294 26160 0 Sep10 ? 01:10:50 tdesl -k 43476
# root 22781 22698 0 00:54 pts/20 00:00:00 grep 11554
Copy the code

Network routing

View route status:

$ route -n
Copy the code

Send a ping packet to the IP address:

$ ping IP
Copy the code

Probe the routing path to the address IP:

$ traceroute IP
Copy the code

Search for the IP address corresponding to the domain name:

$ host domain
Copy the code

Reverse DNS query:

$ host IP
Copy the code

Image download

Download files or web pages directly:

$ wget url
Copy the code

Common options:

  • – limit-rate: indicates the speed limit for downloading
  • -o: specifies a log file. Output is written to the log;
  • -c: resumable data

ftp sftp lftp ssh

SSH login

$ ssh ID@host
Copy the code

Log in to the remote server host using SSH.

FTP/SFTP file transfer

$ sftp ID@host
Copy the code

Log in to server host. The ID is the user name. After logging in to SFTP, you can run the following command to perform further operations:

  • get filename# Download file
  • put filename# upload file
  • lsList all files in the current path on host
  • cdChange the current path on host
  • llsList all files in the current path on the localhost
  • lcdChange the current path on localhost

LFTP sync folder (similar to rsync tool)

$ lftp -u user:pass host
$ lftp user@host:~> mirror -n
Copy the code

Network copy

Upload the local localpath file to the path path of the remote host:

$ scp localpath ID@host:path
Copy the code

Use SSH to traverse the entire file system in the download path to the local localpath:

$ scp -r ID@site:path localpath
Copy the code

User Management Tool

The user

Add user

$ useradd -m username
Copy the code

This command creates an account and user directory /home/username for the user.

After the user is added, set the password:

Passwords are created interactively:

$ passwd username
Copy the code

Delete user

$ userdel -r username
Copy the code

Using userdel without the option will only delete the user, and the user directory will remain in the /home directory. To completely delete user information, use the -r option.

The user’s group

Add a user to a group

Adding a user also adds a user group with the same name. View the group to which the current user belongs:

$ groups
Copy the code

A user can belong to multiple groups. Add a user to a group:

$ usermod -G groupNmame username
Copy the code

Change the root group to which a user belongs (add to a new group and remove from the original group) :

$ usermod -g groupName username
Copy the code

View all system groups

Information about all users and groups in the system is stored in two files: /etc/passwd and /etc/group By default, these two files are accessible to all users:

View all users and permissions:

$ more /etc/passwd
Copy the code

View all user groups and permissions:

$ more /etc/group
Copy the code

The environment variable

Global environment variable Settings

  • /etc/profile
  • /etc/bashrc

Private environment variable Settings in the user directory

  • ~/.profile
  • ~/.bashrc

When logging in to the system and getting a shell process, its reading environment setting script is divided into three steps:

  1. read/etc/profile, and read additional documents based on their contents, such as/etc/profile.dand/etc/inputrc.
  2. read~/.bash_profile, second read~/.bash_login, and finally read~/.profile.
  3. read~/.bashrc.

~/.profile differs from ~/.bashrc:

  • Both have personalization capabilities
  • ~/.profileYou can set the user’s own path, environment variables, etc. It can only be performed once when logged in
  • ~/.bashrcThis is a user-specific configuration document that can set paths, command aliases, and be used once for each shell script execution

Set shortcuts to files and commands that you often enter:

# .bashrc
alias m='more'
log=/opt/applog/common_dir

# .bash_profile
export PS1='$PWD#'
Copy the code

To access the log directory, just type CD $log.

System management

Querying the System Version

View the Linux operating system version:

$ uname -a
$ lsb_release -a
Copy the code

View the Unix operating system version:

$ more /etc/release
Copy the code

Querying Hardware Information

View CPU usage:

$ sar -u 5 10
Copy the code

Querying CPU information:

$ cat /proc/cpuinfo
Copy the code

Check the number of CPU cores:

$ cat /proc/cpuinfo | grep processor | wc -l
Copy the code

Viewing memory information:

$ cat /proc/meminfo
Copy the code

Display memory page size in kbytes:

$ pagesize
Copy the code

Display architecture:

$ arch
Copy the code

Setting the System Time

Display the current system time:

$ date
Copy the code

The resources

  • Linux Tools Quick Tutorial
  • Linux cyc2018 –