According to “Linux Shell Script Guide” according to their own ideas for the summary and content division adjustment. Omit and Web interaction and MySql database, system management monitoring (too rarely used in the current work), and part of the rarely used instructions, but also supplement the common instructions in personal work, more oriented to personal use. Use man to view command help.
A concept,
1.1 the Shell
A Shell, commonly known as a Shell (to distinguish it from a core), is software that “provides the user with an operating interface” (a command parser), receives user commands, and calls the corresponding application program. The shell environment enables users to interact with the core functions of the operating system. Graphical interface/terminal switch: CTRL+ALT+F1~F6 switch, F1 switch to TTy1 /F7 switch to graphical interface /F3 switch to command line (T open terminal in graphical world)
1.2 Shell script
A series of commands that need to be executed are written to the file and then executed by the shell. Script content: Shell scripts usually start with something like #! /bin/bash Start text file. (in the #! Each command or command sequence is separated by a semicolon or newline character; Run the bashrc file using bash script.sh (run the bashrc file using bash)./script.sh (run the bashrc file using bash. When the shell starts, it starts with a set of commands that define Settings such as prompt text, color, etc., such as ~/.bashrc */
1.3 Regular Expressions
Regular expressionIs made up of literal text and symbols with special meaning. We can use them to construct appropriate regular expressions to match text based on specific requirements. Regular expressions are a universal language for matching text. A large number of commands in shell support regular expression matching, such as find to search for files, grep to search for text, etc. The regular expression is not expanded here.
2. Basic grammar
2.1 Variables (read/write/view/environment variables)
The value of each variable in Bash is stored as a string, and you don’t need to declare its type before using it; you just assign it. Write variables: Use var=value to assign values (= cannot contain Spaces, and if values contain Spaces must be quoted), or use var+=add_value to append values to read variables: Add $before the name of a variable to print its contents. Use the set command to view all current variables. Use env to view all environment variables. Environment variables: variables inherited from the parent process that are not defined in the current process. Using the export command, you can set environment variables. Any application executing from the current shell script inherits this variable, such as the environment variable PATH for the executable lookup PATH.
goodboy@ubuntu:~$var=1234 # Note that Spaces cannot be added goodboy@ubuntu:~$echo $var 1234 goodboy@ubuntu:~$var+=5678 goodboy@ubuntu:~$echo $var 12345678 goodboy@ubuntu:~$ echo "we have get var $var" we have get var 12345678 goodboy@ubuntu:~$ echo "we have get var ${var}" we have get var 12345678 goodboy@ubuntu:~$ echo "$var length is ${#var}" 12345678 length is 8Copy the code2.2 Mathematical Calculation
In a Bash shell environment, you can use let, (()), and [] to perform basic arithmetic operations. For more advanced operations, you can use BC, let to perform basic arithmetic operations directly (packet subtraction). When using let, the variable name does not need to be prefixed. The [] operator is used in a similar way to the let command. You can also use (()), except that (()) precedes the variable name. The [] operator is used in a similar way to the let command. You can also use (()), except that (()) precedes the variable name. The [] operator is used in a similar way to the let command. You can also use (()), except that (()) precedes the variable name.
goodboy@ubuntu:~$ num1=4 goodboy@ubuntu:~$ num2=5 goodboy@ubuntu:~$ let sum=num1+num2 goodboy@ubuntu:~$ echo $sum 9 goodboy@ubuntu:~$ let num1-- goodboy@ubuntu:~$ echo $num1 3 goodboy@ubuntu:~$ let num1+=5 goodboy@ubuntu:~$ echo $num1 8 goodboy@ubuntu:~$ sum2=$[num1+num2] goodboy@ubuntu:~$ sum3=$((num1+num2))Copy the code2.3 Expressions (logical operation, comparison of conditions)
(1) Arithmetic comparison: Conditions are usually placed in closed brackets. Be sure to note that there is a space between [or] and operands. **[var−eq0]∗∗ When var-eq 0] ** When var−eq0]∗∗ is true when var is 0. The arithmetic conditions are determined as follows,
-eq equals /-gt equals / -lt equals / -ge equals / -le equals / # [$var1 -ne 0 -a $var2 -gt 2] $var1 -ne 0 -o $var2 -gt 2Copy the code(2) File system related tests: We can use different conditional flags to test different file system related properties
[-f $file_var] : Returns true if the given variable contains a normal file path or filename. [-x $var] : Returns true if the given variable contains an executable file. [-d $var] : Returns true if the given variable contains a directory. [-e $var] : Returns true if the file contained in the given variable exists. [-c $var] : Returns true if the given variable contains the path to a character device file. [-b $var] : Returns true if the given variable contains the path to a block device file. [-w $var] : Returns true if the given variable contains a writable file. [-r $var] : Returns true if the given variable contains a file that is readable. [-l $var] : Returns true if the given variable contains a symbolic link.Copy the code(3) String comparison: When using string comparison, it is better to use double brackets, when using a single bracket will cause an error.
[[$str1 = $str2]] : if str1 = str2, the text is identical, return true. [[$str1 == $str2]] : This is another way to check if strings are equal. [[$str1!= $str2]] : return true if str1 and str2 are different. [[$str1 > $str2]] : Returns true if str1 is larger than str2. [[$str1 < $str2]] : Returns true if str1 has a smaller alphabetic order than str2. [[-z $str1]] : Returns true if str1 contains an empty string. [[-n $str1]] : Returns true if str1 contains a non-empty string.Copy the code(4) logic operations: use logical operators && and | | can easily be combined multiple conditions
if [[ -n $str1 ]] && [[ -z $str2 ]] ;
then
commands;
fi
Copy the code2.4 Statements (Conditional statements/Loop statements)
Conditional statements: We can test with if, if else, and logical operators, using comparison operators to compare data items. In addition, there is a test command that can also be used for testing. The usage of these commands is as follows
if condition;
then
commands;
else if condition; then
commands;
else
commands;
fi
Copy the codeLoop statements: The while loop command is similar to if
while condition;
do
command;
done;
Copy the code2.5 Functions and Parameters
Arg1, arg2, arg2, arg1, arg2… Can be called with arguments. After calling the function, $? The return value (0 for success) is saved;
myfunc() { echo "hello word!" ; } mytest() { echo $1, $2; Echo "$@"; Echo "$*"; # is similar to $@, but the argument is taken as a single entity echo "$#"; Return 0; } myfunc; Mytest arg_A arg_b; # # function with parameter called / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / output as follows / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / goodboy @ ubuntu: ~ $ ./test.sh hello word! arg_a, arg_b arg_a arg_b arg_a arg_b 2Copy the code2.6 Comments — Comments from # to end of line starting with #
GNU/Linux basic commands
Command execution return value: When a command fails and returns, it returns a non-zero exit status. Returns 0 on success. Can exit states from special variables? Run echo immediately after the command is executed. Run echo immediately after the command is executed. Run echo immediately after the command is executed. To print out the exit status.
3.1 Output echo/printf and input read
Echo “XXX”(a newline character is added after each call) use the -n option to ignore the trailing newline character. It can be printed without double quotes, but is encountered; Will think the name is over)
goodboy@ubuntu:~$echo -e "\e[1;31m This is red text \e[0m" This is red text /* \e[1;31 set color to red, \e[0m set color back. Just replace 31 with the desired color code. */Copy the codePrintf can use formatted strings,
goodboy@ubuntu:~$ printf "%-5s %-10s %-4s\n" No Name Mark
No Name Mark
Copy the codeRead: This is used to read text from the keyboard or standard input.
3.2 Setting the command alias/View the command type type/View the history command history
Alias: An alias is a convenient way to save the user the trouble of entering a long sequence of commands. Alias new_command = ‘command sequence.
Alias ls='ls --color=auto' # alias ls='ls --color=auto' #Copy the codeType: You can view the command types. Alias, executable, and shell built-in commands are common.
goodboy@ubuntu:~$ type ls
ls is aliased to `ls --color'
goodboy@ubuntu:~$ type find
find is /usr/bin/find
goodboy@ubuntu:~$ type help
help is a shell builtin
Copy the codeHistory: Displays the history of the recently entered command.
3.3 Obtaining or Setting date date and delay Time of executing the sleep command Time
On Unix-like systems, dates are stored as an integer with the size of the number of seconds that have passed since 00:00 Universal Standard Time (UTC) ① January 1, 1970. This method of timing is called epoch time or Unix time. –date can be used as input. Start_time =$(date +%s)
goodboy@ubuntu:~$date Sun Jun 7 03:35:28 PDT 2020 goodboy@ubuntu:~$date +%s # Unix time 1591526136 goodboy@ubuntu:~$date "+%d %B %Y" 07 June 2020 goodboy@ubuntu:~$ start_time=$(date +%s) goodboy@ubuntu:~$ echo $start_timeCopy the code
Tr/sort sort/eliminate duplicate uniq/split/ %, # take field
Tr: A translate command that replaces, deletes, and compresses repeated characters from the standard input stdin. Tr [options] set1 Set2 maps input characters from stdin from set1 to set2, and then writes the output to stdout. (Common parameter: -d Delete)
Goodboy @ ubuntu: ~ $echo "HELLO! WHO IS THIS" | tr 'a-z' 'a-z' # turn uppercase lowercase HELLO WHO IS THIS goodboy @ ubuntu: ~ $echo "HELLO 123 World 456 "| tr -d '0-9 # delete digital character Hello worldCopy the codeSort versus UNIQ: You can take input from either a specific file or stdin, with a wide range of options and the ability to sort file data in a variety of ways. Uniq is a command that is often used with sort (only for sorted data input). Its purpose is to extract unique (or repeated) lines from text or stdin.
sort unsorted.txt | uniq
Copy the codeSplit: Split files into smaller pieces. Common parameters -b are segmented by size and -l by line number.
goodboy@ubuntu:~$split-l 1 tmp. TXT # Press 1 line SPLT tmp. TXT file goodboy@ubuntu:~$ls tmp. TXT xaa xab xacCopy the codeExtract file fields: With the % operator, you can easily remove the name part from the name. ${VAR%.*}. % is a non-greedy operation. %% is similar to %, but the behavior pattern is greedy. The # operator extracts the extension from the filename and evaluates from left to right, similar to ##.
goodboy@ubuntu:~$ file="tmp.txt"
goodboy@ubuntu:~$ echo ${file%.*}
tmp
goodboy@ubuntu:~$ file2="tmp.a.b.c"
goodboy@ubuntu:~$ echo ${file2%.*} ${file2%%.*}
tmp.a.b tmp
goodboy@ubuntu:~$ echo ${file2#*.} ${file2##*.}
a.b.c c
Copy the code3.5 Job Control View Jobs/Foreground Terminate CTRL + C/background execute & / Background Cut CTRL + Z/Foreground fg/ Background run BG/Schedule task cron
The commands related to jobs are shell commands. Displays the task list and task status in Linux, including tasks running in the background. Task numbers are viewed from the point of view of an ordinary user, while process numbers are viewed from the point of view of a system administrator. A task can correspond to one or more process numbers. A window-like task manager. Jobs (Option)(Parameter) : Additional parameter: -l Displays the process ID. -p Displays only the process id of the task. -n Displays the change of the task status. -r Displays only the tasks in the running status. -s: outputs only stopped tasks. Commands that are being executed in the foreground are terminated by CTRL + C and switched to the background by CTRL + Z. Background work switch fg and BG: use fg % [task number] to switch background to foreground. Bg % [tasks] can be run directly in the background. Background & : If you append & at the end of a command, the task is directly executed in the background.
goodboy@ubuntu:~/ TMP $vi./tmp. TXT # In the vi editing interface CTRL + Z, the execution is switched to the background (stop) [1]+ Stopped vi./tmp. TXT goodboy@ubuntu:~/ TMP $vi ./tmp2.txt # Edit a file with VI, TXT goodboy@ubuntu:~/ TMP $jobs [1]- Stopped vi./tmp.txt [2]+ Stopped Vi./tmp2.txt goodboy@ubuntu:~/ TMP $fg %1 [2] -vi./tmp2.txt ## goodboy@ubuntu:~/ TMP $bg %2Copy the codeScheduled Task CRon: A task performs scheduled work at an agreed time. The CRon server can perform specific tasks based on the time agreed upon in the configuration file. After cron starts, it reads all its configuration files (the global /etc/crontab configuration file, and the scheduled task configuration file for each user), and then cron calls the scheduled task based on the command and execution time. The cron service provides the crontab command to set the cron service. Run the crontab -e command.
# To define the time you can provide concrete values for # minute (m), hour (h), day of month (dom), month (mon), # and day of week (dow) or use '*' in these fields (for 'any').# # Notice that tasks will be started based on the cron's system # daemon's notion of time and timezones. # # Output of the crontab jobs (including errors) is sent through # email to the user the crontab file belongs to (unless redirected). # # For example, you can run a backup of all your user accounts # at 5 a.m every week with: # 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/ # # For more information see the manual pages of crontab(5) and cron(8) # # m h dom mon dow commandCopy the code3.6 Process Control View ps, Dynamic Top, or send signal kill
Process information collection ps: Collects detailed information about processes in the system. This information includes CPU usage, commands being executed, memory usage, process status, and so on. If no parameters are used, ps shows the process running on the current terminal (TTY); Common optional parameters are as follows: -A/e All different processes are displayed. -l More detailed output about the PID. -f More complete output
goodboy@ubuntu:~$ ps
PID TTY TIME CMD
38713 pts/1 00:00:00 bash
38775 pts/1 00:00:00 vi
38777 pts/1 00:00:00 ps
goodboy@ubuntu:~$ ps -ef
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 Jun08 ? 00:00:01 /sbin/init
root 2 0 0 Jun08 ? 00:00:00 [kthreadd]
root 3 2 0 Jun08 ? 00:00:00 [ksoftirqd/0]
root 5 2 0 Jun08 ? 00:00:00 [kworker/0:0H]
Copy the codeDynamic view of Top processes: A list of processes that use the most CPU is output by default. The output is updated every few seconds. A performance analysis tool used in Linux displays the resource usage of each process in the system in real time. -d [seconds] Refresh at intervals of several seconds
Top-08:12:32 UP 1 day, 20:23, 3 Users, Load Average: 0.00, 0.00, 0.00 # 247 Total, 1 running, 244 sleeping, 1 stopped, 1 zombie %Cpu(s): 2.4US, 1.4SY, 0.0ni, 96.2ID, 0.0wa, 0.0hi, 0.0Si, 0.0ST # CPU Usage User mode/Kernel/User priority/Idle/waiting input/Hard interrupt/Soft Interrupt/VM KiB Mem: 2029920 total, 1773960 used, 255960 free, 118092 buffers KiB Swap: 2094076 total, 208 used, 2093868 free. 954876 cached Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1117 root 20 0 376676 81904 42088 S 2.6 4.0 9:56.73 Xorg 38878 goodboy 20 0 29160 3136 R 0.3 0.2 0:00.03 top 1 root 20 0 37360 7948 2772 S 0.3 0.4 0:01. 54 initCopy the codeSend message kill: indicates the process ID of the kill parameter. Signaling is an interprocess communication mechanism in Linux. When a process receives a signal, it responds by executing the corresponding signal handler. Kill -l lists all signal names
goodboy@ubuntu:~$ kill -l 1) SIGHUP 2) SIGINT 3) SIGQUIT 4) SIGILL 5) SIGTRAP 6) SIGABRT 7) SIGBUS 8) SIGFPE 9) SIGKILL 10) SIGUSR1 11) SIGSEGV 12) SIGUSR2 13) SIGPIPE 14) SIGALRM 15) SIGTERM 16) SIGSTKFLT 17) SIGCHLD 18) SIGCONT 19) SIGSTOP 20) SIGTSTP 21) SIGTTIN 22) SIGTTOU 23) SIGURG 24) SIGXCPU 25) SIGXFSZ # goodboy@ubuntu:~$jobs [1]+ Stopped vi tmp.txt goodboy@ubuntu:~$kill -9% 1 # Send termination signal to job 1 goodboy@ubuntu:~$ps PID TTY TIME CMD 38713 pts/1 00:00:00 bash 39151 pts/1 00:00:00 vi 39153 pts/1 00:00:00 ps goodboy@ubuntu:~$ kill -9 39151 # Sends a termination signal to the process with process number 39151Copy the code3.7 Memory Status Free or cat /proc/meminfo/Disk status df du/System information uname
Run the cat /proc/meminfo and free commands to check the system memory status. Run the cat /proc/39316/statm command to check the memory usage of 39316 processes.
goodboy@ubuntu:~$ free
total used free shared buffers cached
Mem: 2029920 1767016 262904 19012 118420 954880
-/+ buffers/cache: 693716 1336204
Swap: 2094076 208 2093868
Copy the codegoodboy@ubuntu:~$cat /proc/meminfo # MemTotal: 2029920 kB # All available RAM size (physical memory minus some reserved and kernel binary size) MemFree: MemAvailable: 1286340 kB # Used memory, some memory used but rebuffed, rebuffed memory plus MemFree Buffers: Vmalloc virtual memory size VmallocUsed: 0 kB VmallocChunk: 0 kBCopy the codeDf and du: two important commands used to collect statistics on disk usage in Linux. Df is short for Disk Free and du is short for Disk Usage. Du additional command: -a recursively outputs the statistics results of the specified directory and all files -h automatically selects appropriate units for easy reading
goodboy@ubuntu:~$du tmp. TXT # Find out the disk space occupied by a file (or files) (default bytes) 12 tmp. TXT goodboy@ubuntu:~$df -h Filesystem Size Used Avail Use% Mounted on udev 978M 4.0k 978M 1% /dev TMPFS 199M 1.1m 198M 1% /run /dev/sda1 18G 4.6g 13G 28% / none 4.0k 0 4.0k 0% /sys/fs/cgroup None 5.0m 0 5.0m 0% /run/lock None 992M 488K 991M 1% /run/ SHM None 100M 48K 100M 1% /run/userCopy the codeUname: Various system information. -n host name -a Displays the kernel version. -r Kernel release version -m Host type.
3.8 Network Control Configuration Ifconfig/Unicom Check Ping/Service analysis netstat
Ifconfig: used to configure and display detailed information about network interfaces and subnet masks. It is usually located in /sbin/ifconfig. Without parameters, the current network interface configurations are listed.
Ifconfig wlan0 192.168.0.80 netmask 255.255.252.0 # Set the IP address and subnet mask fconfig eth0 hw ether 00:1c:bf:87:25:d5 #Copy the codePing: diagnostic tool used to verify the connectivity between two hosts on a network. Ping ADDRESS, ADDRESS can be a host name, domain name, or IP ADDRESS. Ping will send packets continuously, and the response will be printed on the terminal. Press Ctrl+C to stop the ping command
3.4 high order using multiple commands 】 composite pipe ‘|’ transfer xargs and transformation
Pipe symbols’ | ‘: input is usually via stdin or arguments to the command. The output appears either in stderr or stdout. Cmd1 | cmd2 | cmd3 these commands are called filter (filter). We use pipes to connect each filter. The output of CMD1 is passed to CMD2, and the output of CMD2 is passed to CMD3.
Goodboy @ ubuntu: ~ $cat TMP. TXT | grep hello # the TMP. TXT file content passed to grep cat output line search contains hello printf (" hello word \ n ");Copy the codeXarges: some commands can only receive data in the form of command line parameters, and can’t receive the data flow through the stdin, | this time can’t come in handy, xargs to process stdin and transform it into specific command command line parameters. (similar to -exec in the find command), the common argument -n X takes X arguments per execution, and -i {} specifies the replacement string. (The command is executed in a loop.) If there are three arguments, execute three times, each time {} is replaced with the corresponding argument.)
Goodboy @ ubuntu: ~ $find -name "* c" | wc - l # wc actual statistics is the find command output 2 goodboy @ ubuntu: ~ $. Find -name "* c" | xargs wc -l 0./ b.c6. / a.c6 total goodboy@ubuntu:~$cat tmp. TXT Direct the wc content if results for content corresponding to 2 line. / biggest/a.c goodboy @ ubuntu: ~ $cat TMP. TXT | xargs wc - after using xargs l #, The text content is adjusted for the participation All incoming wc statistics 0. / biggest 6. / a.c 6 total goodboy @ ubuntu: ~ $cat TMP. TXT | xargs -i {} {} wc - l # use the -i command, Wc counts one file at a time (no totoal appears) 0./ b.c6./ A.CCopy the codeFour, document related commands and skills
File descriptors: Integers associated with an open file or data stream. 0, 1, and 2 are reserved by the system. 0: stDIN (standard input) 1: stdout (standard output) 2: stderr (standard error)
4.1 File Creation touch/ Delete rm/ link ln/ Move MV/Rename/copy cp
Touch: Modifies the file timestamp. If not, an empty file is created. If the file already exists, the touch command changes all timestamps associated with the file to the current time.
For name in {1.. 100}.txt do touch $name doneCopy the codeLn: Hard links (hard links 0 files are actually deleted) and soft links (symbolic links), similar to shortcuts in Windows. Deleting symbolic links does not affect the original file.
Ln source_file hard_link # hard link ln -s source_file soft_link # soft link readlink # readlinkCopy the codeRename /mv: rename modifies file names using Perl regular expressions; Mv can move a file and rename it with a new name.
Find path-type f-name "*.mp3" -exec mv {} find path-type f-name "*.mp3" -exec mv {} target_dir \; # {} is replaced with multiple single executions, each of which input is the file found by find.Copy the code#! /bin/bash # JPG, image-2.jpg, image-3.jpg, image-4.png, etc. Count =1; for img in `find . -iname '*.png' -o -iname '*.jpg' -type f -maxdepth 1` do new=image-$count.${img##*.} echo "Renaming $img to $new" mv "$img" "$new" let count++ doneCopy the code4.2 File list ls/ Property file/Find/Permission chmod
Ls: displays the file list. -l is an important parameter to list details about files. (Start flag File type: – Common file. D directory. C character device. Block B device. L Symbolic links. S socket. P pipe. The rest can be divided into three groups of three characters each (— — —). Corresponding user rights, user group rights, and user rights. Read permission r, write permission W, execute permission x, – indicates no corresponding permission), -a lists hidden files.
goodboy@ubuntu:~$ ls -al total 68 drwxr-xr-x 4 goodboy goodboy 4096 Jun 8 09:32 . drwxr-xr-x 4 root root 4096 Jun 6 07:48.. -rw-rw-r-- 1 goodboy goodboy 92 Jun 7 02:44 a.c -rw-rw-r-- 1 goodboy goodboy 12 Jun 8 09:32 a.txt -rw------- 1 goodboy goodboy 80 Jun 6 08:18 .bash_history -rw-r--r-- 1 goodboy goodboy 220 Jun 6 07:48 .bash_logout -rw-r--r-- 1 goodboy goodboy 3637 Jun 6 07:48 .bashrcCopy the codeChmod: sets file permissions. Permissions can be set using octal numbers, r– = 4-w – = 2 –x = 1. You can add permissions to users, user groups, and other users by using +, delete permissions by using -, or directly set permissions (u user permissions g user group permissions o other entity permissions).
Chmod 764 filename # RWX rw-r -- equal to 764 chmod u= RWX g=rw o=r filename chmod o+x filenameCopy the codeFile: prints the file type information. (-b Prints information about the file type excluding the file name)
goodboy@ubuntu:~$ file /bin/grep
/bin/grep: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=9780fb82e027800459e20af35f65055688a3c031, stripped
goodboy@ubuntu:~$ file a.c
a.c: C source, ASCII text
Copy the codeFind: Traverses down the file hierarchy, matches matched files, and performs corresponding operations. -path Matches the path as a whole. -maxdepth n Specifies the depth of the search layer. -type by type. Negative arguments can also be used! Reverse matching, -o/-a multi-condition combination, -exec** append execution command (-exec command {} \; Among them; The forward slash is prefixed in case of explanation), similar to xargs.
Find. -name "*.txt" # # find. (-name "*.txt" -o-name "*.c" \) # -path "* HWJ *" # Start from the current position to search for files/folders containing HWJ strings in the complete path find. -type d -print # List all directories find. -type f -atime +7 # list all files that have been accessed for more than 7 days. -7 last 7 days. Find. -type f-size-2k # find. -type f-exec file {} \; Find each file and perform the file query action on them (one at a time)Copy the code4.3 File Contents View all cat files/By page more/The first few lines head/ the last few lines tail/Binary OD
Cat :cat file1 file2 file3 Prints files to standard output. (Common additional command: -s delete excess blank line -n with line number). In contrast to cat, head and tail can view only a small portion of the file’s contents. The default value is 10 lines, and the contents can be specified by -n.
4.4 File Statistics WC (Line, Word, Or Character)
Wc: counts the number of lines, words, and characters in a file. A separate type can be printed with the options -l number of lines, -w number of words, and -c number of characters.
goodboy@ubuntu:~$ find . -name "*.c" | xargs wc
0 0 0 ./b.c
6 11 92 ./a.c
6 11 92 total
Copy the code4.5 Files and Redirection <, >, and >>
Standard input (STDIN), standard output (STdout), and standard error (stderr) are redirected to file common operations through content filtering. Input redirection [n defaults to 0]
[|] word: is the output of the shell content print out from the window to a file (clears the original content). Extended output redirection [n default 1]>> Word: redirects the shell command output stream to the back of the file without deleting the contents of the original file;
goodboy@ubuntu:~$ echo "this is a sample text" > tmp.txt
goodboy@ubuntu:~$ cat tmp.txt
this is a sample text
goodboy@ubuntu:~$ echo "this is a second sample text" >> tmp.txt
goodboy@ubuntu:~$ cat tmp.txt
this is a sample text
this is a second sample text
Copy the codeException output redirection and redirection binding: CMD > file is the same as CMD 1 > file (exception information is still output to the terminal). If you want to redirect abnormal output to a file (error information is often exported during compilation), you can use CMD 2 > file. CMD 2>stderr.txt 1>stdout. TXT can be entered into two files respectively. TXT or CMD &> output. TXT (redirect binding, converting stderr to stdout so that both stderr and stdout are redirected to the same file)
goodboy@ubuntu:~$ ls +
ls: cannot access +: No such file or directory
goodboy@ubuntu:~$ ls + 2> tmp.txt
goodboy@ubuntu:~$ cat tmp.txt
ls: cannot access +: No such file or directory
Copy the codeCMD 2>/dev/null: CMD 2>/dev/null: CMD 2>/dev/null: CMD 2>/dev/null: CMD 2>/dev/null
4.6 File Comparison Comm Difference Diff and patch patch
Comm: Can be used to compare two files, and can be used for intersection, difference, and difference set operations. (Comm must use a sorted file as input.) Without an option, the program generates three columns of output. The first column contains rows unique to file 1, the second column contains rows unique to file 2, and the third column contains rows common to both files. Option command: -1 does not output lines unique to file 1. -2 does not output lines unique to file 2. -3 does not output lines common to two files
goodboy@ubuntu:~$ cat a.txt 123 456 789 goodboy@ubuntu:~$ cat b.txt 234 456 goodboy@ubuntu:~$ comm a.txt b.txt 123 234 456, 789,Copy the codeDiff vs. Patch: Diff generates two file differences and the -u option is used to generate integrated output. It is more readable and easier to see the difference between two files, starting with + for newly added lines and starting with – for deleted lines. Use the patch command to apply the changes to any file. When applied to version1.txt, we get version2.txt; When applied to version2.txt, you get version1.txt.
Diff -u version1.txt version2.txt > version.patch # Patch files can be generated by redirecting the output of diff to a file to produce patch-p1 version1.txt < version.patch The contents of #version1.txt are now exactly the same as the contents of verson2.txtCopy the code4.7 Archive Packaging and Compression tar
Tar: Archive files, save multiple files and folders into a single file, and retain all file attributes, such as owner and permissions. Additional argument: -v verbose mode to get more detail when archiving or listing archived files. Tar -cf output.tar file1 file2 file3 folder1.. Add two archive files: tar -af file1.tar file2.tar Query files in the archive: tar -tf archive.tar Extract contents: Tar -xf archive.tar -c /path/to/extraction_directory #
Compress tar archive files: The tar command can only be used to archive files. It does not have the compression function. For this reason, most users use some form of compression when working with archive files. -j: bunzip2 (file.tar.bz2) -z: gzip (file.tar.gz) — lzMA: LZMA (file.tar.lzma)
goodboy@ubuntu:~$ tar -zcvf ./tmp.tar.gz tmp.txt tmp.txt goodboy@ubuntu:~$ ls -l tmp.tar.gz tmp.txt -rw-rw-r-- 1 goodboy goodboy 169 Jun 9 09:45 tmp.tar.gz -rw-rw-r-- 1 goodboy goodboy 9021 Jun 9 09:44 tmp.txt goodboy@ubuntu:~$ tar -xzvf ./tmp.tar.gz -C tmp/ tmp.txtCopy the code4.8 Creating directories mkdir/ Deleting rmdir/Switching CD/Printing directories to enter tree
Tree: The tree command prints files and directories in a graphical tree structure. (This command is usually not included in Linux distributions, so you need to install it yourself), optional: -h Prints the size of both the file and the directory
4.9 [Advanced Skills] Transferring files over the Network FTP/setP/rsync/ SCP
5. Text-related commands and skills
Internal Field Separator (IFS) : Delimiter that separates a single data stream into different data elements. Internal field delimiters are delimiters used for a specific purpose. IFS is an environment variable that stores delimiters (the default delimited string).
5.2 File Search Text grep
Grep: Searching in file \ standard output is text processing and can accept regular expressions. Additional parameters: -n with line numbers -r recursion -o Input only matched text parts -I ignore case -e Multiple pattern matches –include specify files –exclude exclude files -l Output matching files -e Use extended regular expressions (or use egrep as an example) -v to reverse the matching result, The rest of the line -c simply counts the number of rows that match and prints the lines that match nearby: the content after -a, the content before -b, and the content before and after -c.
goodboy@ubuntu:~$ echo this is a line. | grep -E -o "[a-z]+\."
line.
goodboy@ubuntu:~$ echo -e "1 2 3 4\nhello\n5 6" | egrep -o "[0-9]" | wc -l
6
goodboy@ubuntu:~$ echo this is a line of text | grep -e "this" -e "line" -o
this
line
goodboy@ubuntu:~$ grep -nr "main" --include '*.c'
a.c:2:int main(void) {
Copy the codeGrep’s silent output: Sometimes you don’t want to look at the matching string, just to see if it was successfully matched. This can be done by -q. The command returns 0 on success and a non-zero value on failure.
goodboy@ubuntu:~$ grep -q "main" a.c
goodboy@ubuntu:~$ echo $?
0
goodboy@ubuntu:~$ grep -q "xxmain" a.c
goodboy@ubuntu:~$ echo $?
1
Copy the code5.3 Cut text and Merge paste
Cut: Tool for slicing text into columns. -b for bytes -c for characters -f is used to define fields (to specify delimiters for fields, use the -d option). N minus from the NTH to the tail/N minus M from the NTH to the MTH / -m the first to the MTH.
Cut -f 2,3 filename # $cat delimited_data. TXT No; Name; Mark; Percent 1; Sarath; 45. 90 2; Alex; 49. 98 3; Anu; 45. 90 $ cut -f2 -d";" delimited_data.txt Name Sarath Alex AnuCopy the code5.4 Text Replacement and Modification Stream editor sed
Sed is short for Stream Editor. With regular expressions, you can do a lot of work. For example: Replace a string in a given text: sed ‘s/pattern/replace_string/g ‘file #s is used as a replacement. If g is not added, only the first newline is mentioned. Delete blank line: sed ‘/^$/d’ file #d for delete in file: sed ‘s/PATTERN/replacement/’ -i filename #-i for modify in file. Matched string tag & : You can match style strings with the & tag, so that you can use the matched content when replacing strings.
Goodboy @ ubuntu: ~ $echo this is an example | sed 's / \ \ w + / [&] / g' # match each word, and then we replace it with [&] [this] [is] [an] [example]Copy the codeSubstring matching flag (\1) : can match any part of a given pattern. (Pattern) is used to match substrings. The pattern is wrapped in () escaped with a slash. For the first matched substring, the corresponding flag is \1, for the second matched substring, \2, and so on.
goodboy@ubuntu:~$ echo this is digit 7 in a number | sed 's/digit \([0-9]\)/\1/' this is 7 in a number goodboy@ubuntu:~$ echo seven EIGHT | sed 's/\([a-z]\+\) \([A-Z]\+\)/\2 \1/' EIGHT sevenCopy the code