This article is from MIT’s “The Missing Lesson in Computer Education” course, which covers the command line, the use of a powerful text editor, and the many features provided by a version control system. Chinese Course Homepage:https://missing-semester-cn.github.io/.

In this lesson, I introduce you to the basics of bash as a scripting language, as well as some of the most common Shell tools.

  • Variable assignment:foo=bar, note that no Spaces can be added between them
  • The string in Bash passes' 和 "The delimiter is defined, but their meaning is different. In order to'The string defined isLiteral string, where variables are not escaped while"Defined stringThe value of the variable is replaced.
  • Function writing: use"$1"Get the value of the variable
mcd () {
    mkdir -p "$1"
    cd "$1"
}

  • Special variables in bash

    • $0– the script name
    • The $1 to $9 – Parameters to the script.The $1Is the first parameter, and so on.
    • $@– All parameters
    • $#– Number of parameters
    • $? – The value returned from the previous command
    • $$– The process identifier of the current script
    • !!!!! – Complete previous command, including parameters. Common application: when you fail to execute a command because of insufficient permissions, can usesudo !!Try again.
    • The $_– The last argument to the previous command. If you’re using an interactive shell, you can do so by pressingEscThen type. To get the value.
  • Special variables can be paired with short-circuit operators to interrupt a program
  • Command substitution: Gets the output of a command as a variable when passed$( CMD )This is the way to do itCMDThe output of this command will be replaced$( CMD ). For example, if you executefor file in $(ls), the shell will first callls, and then iterate over the resulting return values.
  • Process substitutions: A similar feature that’s less popular is process substitution,<(CMD)CMD is executed and the results are output to a temporary file, and the<( CMD )Replace with a temporary file name. This is useful when we want the return value to be passed through a file rather than STDIN. For example,diff <(ls foo) <(ls bar)It will show the folderfoo 和 barThe difference between the files in.
  • Program example, variable is filename, if the filename contains “foobar”, do not operate, otherwise add “foobar”
#! /bin/bash echo "Running program $0 with $# arguments with pid $$" echo "Running program $0 with $# arguments with pid $$" for file in "$@"; Do grep foobar "$file" > /dev/null 2> /dev/null # If the mode is not found, grep exit status is 1 # We redirect the standard output stream and the standard error stream to null, Because we don't care about that information. Echo "File $File does not have any foobar, adding one" echo "# foobar" >> "$File "fi done

  • This is useful for batch file processing: bash wildcard (globbing).

    • Wildcards – when you want to match with wildcards, you can use them separately. And * to match one or any character.
    • Curly braces{}– When you have a list of commands that contain a common substring, use curly braces to automatically expand the commands. This is handy when moving or converting files in bulk.
Convert Image.{PNG, JPG} # will expand to Convert Image.png image.jpg cp /path/to/project/{foo,bar,baz} /path/to/project/foo.sh /path/to/project/bar.sh /path/to/project/baz *. Py and *. Sh files will be moved mkdir foo bar # foo/h, bar/a, bar/b, ... Bar /h These files touch {foo,bar}/{a.. H} touch foo/x bar/y # Compare folders foo and bar containing different files diff <(ls foo) <(ls bar) # output # < x # -- # > y
  • ShellCheck is a tool to check for errors in shell scripts. There are programs and plugins
  • Find help:manProgram, but the output is too long,You can usetldrProgram, output a few examples (very useful!
  • Find the file:find 和 fd ， fdThe default support for regular lookup is more intuitive.locateUsing a database is a faster way to search, but the drawback is that you can only search by the file name.
  • Find the code:grepThere are a lot of options, which makes it a very versatile tool. Among them I often use are-C ：Get the Context of the search result;-vThe result is Invert, or a mismatch is printed. For example,grep -C 5It will print five lines before and after the match. Used when a large number of files need to be searched-RIt recursively goes to the subdirectory and searches for all the text files. However, there are many ways we can do thisgrep -RMake improvements, such as ignoring them.gitFolders, using multiple CPUs, etc. So there are plenty of alternatives: Ripgrep (RG) is a popular one because of its speed and intuitive usage. Examples are as follows:
Rg-u -- files-unmatched "^#!" rg-u -- files-unmatched "^#!" rg-u -- files-unmatched "rg-u -- files-unmatched" rg-u -- files-unmatched "rg-u -- files-unmatched" # Find all the foo strings and print the next 5 lines rg foo-a 5 # Print the matching statistics (number of matched lines and files) rg --stats PATTERN

  • To find the command

    • History accesses the history command entered in the shell
    • For most shells, you canCtrl+RA retrospective search of the command history is performed. knockCtrl+RThen you can enter a substring to match and look up the history command line. Repeatingly pressing the button will loop through all search results. inzshThis can also be done using the arrow keys up or down.

  • navigation

    • ranger
    • AutoJump, command j

Assignments section

  1. The ls command

    1. Display all files:ls -a
    2. Output in a comprehensible format:ls -lh
    3. Sort by most recent access order:ls -tl
  2. Write two bash functions, Marco and Polo, to do the following. Whenever you execute Marco, the current working directory should be saved in some form. When you execute Polo, CD should go back to the directory where Marco was executed, no matter what directory you are in. To debug easily, you can write the code in a separate file called marco.sh and (re) load the function with the source marco.sh command.
macro () {
    current="$(pwd)"
    echo "$current saved to cache"
}

polo () {
    cd "$current"
    echo "jump to $current"
}
  1. Suppose you have a command that rarely goes wrong. So in order to be able to debug when something goes wrong, a lot of time is spent reproducing the error and capturing the output. Write a bash script, run the following script until it fails, log its standard output and standard error stream to a file, and output everything at the end. Bonus: Report how many times the script was run before it failed.
#! /usr/bin/env bash count=0 while true; do n=$((RANDOM % 100)) if [[ n -eq 42 ]]; then echo "Something went wrong" echo >&2 "The error was using magic numbers" echo "Program successfully ran $count times\n" exit 1 fi ((count += 1)) echo "ran successfully" done echo $count echo "Everything went according to plan"

Sh [capture. Sh](http://capture.sh/) 1> output 2> error; cat output; cat error

  1. Your task is to write a command that recursively finds all the HTML files in the folder and compresses them into a ZIP file. Note that your command should execute correctly even if the filename contains Spaces (hint: check the xargs argument -d for this issue). If you are using MacOS, please note that the default BSD find is different from the one in GNU Coreutils. You can add the -print0 option for find and the -0 option for xargs. As a Mac user, you need to be aware that there are differences between the command-line tools that come with the Mac and their GNU counterparts; If you want to use the GNU version of the tool, you can also use BREW to install it.
find *.html | xargs -d zip compressed.zip
  1. (Advanced) Write a command or script to recursively find the most recently used files in the folder. More generally, could you list the files by the last time they were used?