GitHub 60000+ Star: The art of command lines

Being proficient at the command line is a skill that is often overlooked or thought to be difficult to master, but it actually increases your flexibility and productivity as an engineer. This article is a summary of some of the command line techniques I discovered while working on Linux. Some techniques are very basic, while others are quite complex or even obscure. This article isn’t long, but once you’ve mastered all the techniques listed here, you’ll learn a lot about the command line.

This article is the result of many authors and translators. Some of the content here first appeared on Quora, but has since migrated to Github, with many improvements made by experts. If you see any errors or improvements in this article, please contribute.

preface

Scope of coverage:

• This article will be useful not only for those new to the command line, but also for those who have experienced it. This article aims to doWide coverage(all the important stuff),specific(give specific examples of the most commonly used), andconcise(Avoid redundant content or details that can be easily found elsewhere). In specific application scenarios, the content in this article is basic or can help you save a lot of time.
• This article is written primarily for Linux, but there are also operating system references in the OS X Only section and Windows Only section. Aside from these two chapters, most of the rest can be applied to other UniX-like systems or OS X, or even Cygwin.
• This article focuses primarily on interactive Bash, but there are many techniques that can be applied to other shell and Bash scripts.
• In addition to the “standard” Unix commands, this article includes some commands that depend on a particular package (if they are valuable enough).

Notes:

• In order to display as much as possible on one page, some specific information can be found on the referenced page. We’re sure you know how to use Google or other search engines to get more detailed information. Some of the commands in this article require you to use themapt-get.yum.dnf.pacman.pip 或 brew(and other appropriate package managers) to install dependent programs.
• If you run into problems, try using Explainshell to get an explanation of the commands, parameters, pipes, etc.

basis

• Learn the basics of Bash. Specifically, type man bash on the command line and browse through the full text at least once; It’s easy to understand and not too long. Other shells may be useful, but Bash is powerful enough to be almost always available (it will be convenient on your own device if you only learn ZSH, Fish, or other shells, but relying too much on them will be inconvenient, For example, when you need to work on a server.

• Be familiar with at least one text-based editor. In general, Vim (vi) is your best choice, since Vim is the most useful tool for editing text on the terminal (even better than Emacs, large ides, or cool editors in most cases).

• Learn how to read documents using the man command. Learn to use apropos to find documents. Knowing that some commands do not correspond to executable files but are built into Bash, you can use the help and help -d commands to obtain help information. You can use the type command to determine whether the command is an executable, a shell built-in command, or an alias.

• Learn how to use > and < to redirect the output and input, learn to use | to redirect the pipeline. Understand that > overwrites the output file and >> is added at the end of the file. Understand stDout and STderr.

• Learn to use the wildcard * (and maybe? And […]. ) and citations and the difference between ‘and “in citations (some specific examples later).

• Familiar with Bash task management tools: &, Ctrl-Z, Ctrl-C, Jobs, FG, BG, kill, etc.

• Learn how to use SSH to log in to the remote cli. It is better to know how to use the ssh-agent and ssh-add commands to implement basic password-less login authentication.

• Learn basic file management tools: Ls and ls -L (understand the meaning of each column in ls -L), less, head, tail and tail -f (even less + f), ln and ln -s (understand the difference between hard and soft links), chown, chmod, Du (Overview of hard disk usage: du-hs *) For file system management, learn df, mount, fdisk, MKFS, LSBLK. Know what an inode is (related to commands like ls -i and df -i).

• Learn basic network management tools: IP or ifconfig, dig.

• Learn and use a version control management system such as Git.

• Be familiar with regular expressions, and learn to use grep/egrep. The -i, -o, -v, -a, -b, and -c arguments are common and worth studying carefully.

• Learn to use apt-get, yum, DNF, or Pacman (depending on which Linux distribution you use) to find and install packages. And make sure you have PIP in your environment to install python-based command-line tools (some of the programs mentioned below can be easily installed using PIP).

Daily use

• In Bash, you can automatically complete parameters by pressing Tab, and use Ctrl-R to search the command line history (after you press the key, Enter the keyword to search, press Ctrl-R repeatedly to look back for matches, press Enter to execute the currently matched command, Pressing the right arrow key, on the other hand, puts the match in the current line and does not execute directly for modification).

• In Bash, you can press Ctrl-W to delete the last word you typed, Ctrl-U to delete the line before the cursor, Alt-b and Alt-F to move the cursor by word, Ctrl-A to move the cursor to the beginning of the line, Ctrl-E to move the cursor to the end of the line, and ctrl-W to move the cursor to the end of the line. Ctrl-k can delete the cursor to the end of the line, ctrl-L can clear the screen. Type man readline to see the default shortcut keys in Bash. There are many contents, such as Alt -. That circularly moves the previous argument, and Alt -* can expand the wildcard.

• If you like, you can use the vi-style shortcut by running set -o vi, and run set -o emacs to change it back.

• To make editing long commands easier, after setting your default EDITOR (such as export EDITOR=vim), Ctrl-X ctrl-E will open an EDITOR to edit the currently typed command. In vi style, the shortcut is escape-v.

• Type history to view the command line history, then use! N (n is the command number) can be executed again. There are many abbreviations, and probably the most useful one is! $, which is used to refer to the last parameter typed, and!! You can refer to the last command you typed (see “HISTORY EXPANSION” on the man page). However, you can also use the shortcut keys Ctrl-R and Alt -. To implement.

• Run the CD command to change the working path. Enter CD ~ to go to the home directory. To access files in your home directory, use the prefix ~ (for example ~/.bashrc). In the sh script, the environment variable $HOME is used to refer to the path to the HOME directory.

• Go back to the previous working path: CD -.

• If you change your mind in the middle of typing a command, press Alt -# to add # to the beginning of the line as a comment and press Enter to execute (or ctrl-A, #, Enter). By doing so, you can easily recover the command you just typed halfway through using the command line history later.

• Use xargs (or PARALLEL). They are very helpful. Note that you can control the number of arguments per line (-l) and the maximum number of parallelisms (-p). If you’re not sure if they will work as you want, use the Xargs Echo to check first. Also, it is convenient to use -i {}. Such as:

      find . -name '*.py' | xargs grep some_function
      cat hosts | xargs -I{} ssh root@{} hostname
Copy the code

• Pstree-p presents the process tree in an elegant way.

• Use pgrep and pkill to find or signal processes by name (the -f argument is usually useful).

• Know the kinds of signals you can send to the process. For example, use kill -stop [pid] to STOP a process. Use man 7 signal to view the detailed list.

• Use nohup or disown to keep a background process running.

• Use netstat -lntp or ss-plat to check which processes are listening on the port (TCP port is checked by default; Add -u to check UDP port) or lsof -itcp-stcp: LISTen-p -n (this can also run on OS X).

• Lsof to view open sockets and files.

• Use uptime or W to see how long the system has been running.

• Use alias to create shortcuts to common commands. For example: alias ll=’ ls-latr ‘Creates a new command alias ll.

• You can save aliases, shell options, and common functions in ~/.bashrc, as shown in this article. Doing so allows you to use your Settings in all shell sessions.

• Save the environment variable Settings and the commands to execute at login to ~/.bash_profile. Separate configuration files are required for shells launched from a graphical interface and shells launched from cron.

• To synchronize your configuration files (such as.bashrc and.bash_profile) across several computers, Git can be used.

• Be careful when variables and filenames contain Spaces. Bash variables are enclosed in quotes, such as “$FOO”. Use – 0 or -print0 option as far as possible in order to use NULL to separate file names, such as the locate the pattern | 0 xargs 0 ls – al or find / -print0 -type d | xargs 0 ls – al. If the filename accessed in the for loop contains null characters (Spaces, tabs, etc.), just use IFS=$’\n’ to set the internal field separator to a newline.

• In Bash scripts, use set-x to debug the output (or its variant, set -v, which records the original input, including redundant arguments and comments). Use strict mode whenever possible: Use set -e to make the script exit on an error rather than continue; Use set -u to check whether unassigned variables are used; Try set-o pipeFail, which monitors errors in the pipeline. Use trap to detect ERR and EXIT when many scripts are involved. It is a good practice to write this at the beginning of a script file. This will enable it to detect some errors and interrupt the program and output information when an error occurs:

      set -euo pipefail
      trap "echo 'error: Script failed: see failed command above'" ERR
Copy the code

• In Bash scripts, the subshell (use parentheses(...).) is a convenient way to organize parameters. A common example is to temporarily move a work path, as follows:

      # do something in current dir
      (cd /some/other/dir && other-command)
      # continue in original dir
Copy the code

• In Bash, variables can be extended in many ways. ${name:? Error message} is used to check whether the variable exists. In addition, when the Bash script requires only one argument, you can use the code input_file=${1:? Usage: $0 input_file}. Use the default value when the variable is empty: ${name:-default}. If you want to add another (optional) argument to the previous example, you can use code like output_file=${2:-logfile}. If $2 is omitted, its value is null, and output_file is set to logfile. I =$(((I + 1) % 5)) Sequence: {1.. 10}. Truncate strings: ${var%suffix} and ${var#prefix}. For example, assuming var=foo.pdf, echo ${var%.pdf}.txt will output foo.txt.

• Use parentheses to extend ({… }) to reduce similar text input and automate text composition. This can be useful in some cases, such as mv foo.{TXT, PDF} some-dir (move two files at once), Cp somefile{,.bak} (will be extended to cp somefile somefile.bak) or mkdir -p test-{a,b,c}/subtest-{1,2,3} (will be extended to all possible combinations and create a directory tree).

• You can treat the output as a file by using <(some command). For example, compare the local file /etc/hosts to a remote file:

      diff /etc/hosts <(ssh somehost cat /etc/hosts)
Copy the code

• When writing a script, you might want to put your code in curly braces. Without the closing parenthesis, the code will fail to execute because of a syntax error. If your script is going to be shared online for others to use, this is a good way to write it, because it prevents incomplete code from being downloaded and executed.

{
      # Write code here
}
Copy the code

• Learn about “here Documents” in Bash, such as cat <

• In Bash, redirect both standard output and standard error: some-command >logfile 2>&1 or some-command &>logfile. In general, it is a good practice to add

• Use MAN ASCII to view an ASCII table with hexadecimal and decimal values. Man Unicode, MAN UTF-8, and Man Latin1 help you understand common encoding information.

• Using Screen or tmux to use multiple screens is especially useful when you are using SSH (saving session information). Byobu provides them with more information and easy-to-use management tools. Another lightweight session persistence solution is DTACH.

• In SSH, it’s useful to know how to start a tunnel using -L or -d (and occasionally -r), such as when you need to access a Web page from a remote server.

• It may be useful to make a few minor optimizations to SSH Settings, such as the ~/. SSH /config file, which contains options to prevent connection disconnections, compress data, multichannel, and so on in a particular network environment:

      TCPKeepAlive=yes
      ServerAliveInterval=15
      ServerAliveCountMax=6
      Compression=yes
      ControlMaster auto
      ControlPath /tmp/%r@%h:%p
      ControlPersist yes
Copy the code

• Some of the other SSH options are security-related and should be used with caution. For example, you should only enable StrictHostKeyChecking=no, ForwardAgent=yes on trusted networks.

• Consider using MOsh as an alternative to SSH, which uses the UDP protocol. It avoids connection interruptions and requires less bandwidth, but it needs to be configured on the server.

• To get access to files in octal (usually required when changing system Settings, but ls is less useful and often messed up), you can use code like this:

      stat -c '%A %a %n' /etc/timezone
Copy the code

• Use percol or FZF to interactively select a value from another command output.

• You can use FPP (PathPicker) to interact with files based on the output of another command, such as git.

• To expose all files (and subdirectories) in the current directory on your Web server to all users on your network, use: Python -m SimpleHTTPServer 7777 (use port 7777 and Python 2) or Python -m http.server 7777 (use port 7777 and Python 3).

• Execute the command as another user, using sudo. By default, the command is executed as the root user. Use -u to specify other users. Use -i to log in as that user (with _ your own _ password).

• Switch the shell to another user, using su username or sudo-username. Add – To make the environment the same as the one you log in to as the user. If the user name is omitted, the default value is root. To switch to the user, you need to enter _ user’s _ password.

• Understand the 128K limit of the command line. When wildcard characters are used to match a large number of file names, the “Argument list is too long” error message is often encountered. (Substituting find or xargs in this case usually solves the problem.)

• When you need a basic calculator, you can use the Python interpreter (even if you want to use Python). Such as:

> > > 2 + 3 5Copy the code

Document and data processing

• To find a file by file name in the current directory, use a command like find. -iname ‘*something*’. Find files by file name in all paths, using Locate Something (note that updatedb may not index newly created files, so you may not be able to locate unindexed files).

• Use AG for source code or data file retrieval (grep -r can do the same, but AG is more advanced).

• To convert HTML to text: lynx-dump-stdin.

• For Markdown, HTML, and all the other document formats, try Pandoc.

• When you’re dealing with tricky XML, XMLStarlet is a relic from the oldies.

• Use JQ to handle JSON.

• Use shyamL to process YAML.

• To handle Excel or CSV files, csvKit provides in2CSV, csvCut, csvJoin, csvgrep, and other easy-to-use tools.

• S3cmd is a handy tool when you’re dealing with Amazon S3 and S4CMD is much more efficient. Aws and SAWS provided by Amazon are the foundation of other AWS related work and are worth learning.

• Learn how to use sort and UNIq, including the -u and -d arguments to UNIq, as described in the one-line script section below. In addition, you can learn about Comm.

• Learn how to use cut, paste, and join to change files. Many people use cut but forget join.

• Learn how to use WC to count new lines (-L), characters (-m), words (-w), and bytes (-c).

• Learn how to use the tee will be copied into the standard input file even standard output, such as ls – al | tee file. TXT.

• For complex calculations such as grouping, inversions, and other statistical analyses, consider using Datamash.

• Note that the language Settings (Chinese, English, etc.) have subtle effects on many command-line tools, such as sorting order and performance. Most Linux installations set LANG or other related variables to local Settings. Be aware that the results of the sort may change when you change the language Settings. Understand that internationalization can make sort or other commands run many times less efficiently. In some cases (such as collection operations) you can safely use export LC_ALL=C to ignore internationalization and determine the order by bytes.

• You can specify the environment of a command by calling it with the environment variable at the beginning of the command. For example, TZ=Pacific/Fiji date can get the time of Fiji.

• Learn how to use AWK and SED for simple data processing. See the one-liners fetch example.

• To replace a string that appears in one or more files:

      perl -pi.bak -e 's/old-string/new-string/g' my-files-*.txt
Copy the code

• usereprenTo rename files in batches, or search for replacement contents in multiple files. (SometimesrenameThe command can also be renamed in bulk, but be aware that it does not work exactly the same across Linux distributions.)

      Foo -> bar: foo -> bar
      repren --full --preserve-case --from foo --to bar .
      # restore all backup files whatever. Bak -> whatever:
      repren --renames --from '(.*)\.bak' --to \ '1' *.bak
      Use rename to do the above (if available) :
      rename 's/\.bak$//' *.bak
Copy the code

• According to the description on the man page,rsyncIs a fast and very flexible file copy tool. It’s known for synchronizing files between devices, but it’s actually useful in local situations as well. If security Settings allow, usersyncInstead ofscpCan implement file continuation, rather than starting from scratch. It also removes a large number of filesThe fastest way toOne of:

mkdir empty && rsync -r --delete empty/ some-dir && rmdir some-dir
Copy the code

• To get the current progress when copying a file, use PV, pycp, progress, rsync –progress. If the copy is block block copy, use dd status=progress.

• Shuf allows you to shuffle the contents of a file in units of behavior or to randomly select multiple lines from a file.

• Learn the parameters of sort. When displaying numbers, use -n or -h to display more readable numbers (such as the output of du -h). Understand how keywords work when sorting (-t and -k). For example, notice that you need -k1, 1 to sort by the first field only, whereas -k1 means to sort by the whole row. Stable sort (sort-s) can be useful in some situations. Give priority to with the second domain, for example, keywords, keyword ranking first field as the time, you can use the sort – k1, 1 | sort – s – k2, 2.

• If you want to write a TAB on the Bash command line, press Ctrl-V [TAB] or type $’\t’ (the latter is probably better, since you can copy and paste it).

• The standard source code comparison and merge tools are Diff and Patch. Use Diffstat to view the change overview data. Notice that diff -r is valid for the entire folder. Use the diff – r tree1 tree2 | diffstat view change statistics. Vimdiff is used to compare and edit files.

• For binary files, use HD, hexdump, or XXD to display them in hexadecimal, and use BVI, hexEdit, or Biew for binary editing.

• Also for binaries, strings (including tools like grep) can help you find specific bits in the binaries.

• Make binary difference files (Delta compression) using xdelta3.

• Use iconV to change the text encoding. For more advanced functionality, you can use UCONv, which supports some advanced Unicode functionality. For example, this command removes all accents:

      uconv -f utf-8 -t utf-8 -x '::Any-Lower; ::Any-NFD; [:Nonspacing Mark:] >; ::Any-NFC; ' < input.txt > output.txt
Copy the code

• Split files can be split (split by size) and csplit (split by mode).

• To manipulate date and time expressions, you can use dateAdd, Datediff, strptime, and other tools in dateutils.

• Use zless, zmore, zcat, and zgrep to operate compressed files.

• File properties can be set via chattr, which is at a lower level than file permissions. For example, in order to protect the file was accidentally deleted, you can use the immutable flag: sudo chattr + I/critical/directory/or/file

• Use getFacl and setfacl to save and restore file permissions. Such as:

   getfacl -R /some/path > permissions.txt
   setfacl --restore=permissions.txt
Copy the code

• To efficiently create empty files, usetruncate(createSparse file),fallocateFor ext4, XFS, BTRF and OCFs2 file systems,xfs_mkfile(for almost all file systems, included in the xfsprogs package),mkfile(for UniX-like operating systems such as Solaris and Mac OS).

System debugging

• Curl and curl -I can be easily applied to Web debugging, as can their cousin wget, or try the more trendy Httpie.

• Get CPU and hard disk usage, usually using top (hTOP is better), iostat, and ioTOP. Iostat-mxz 15 gives you basic information about the CPU and the performance of each disk partition.

• Use netstat and ss to view network connection details.

• Dstat is very useful if you want to get a rough idea of the state of the system. However, to get a deeper overview of the system, use Glances, which provides you with some system-level data in a terminal window.

• To understand memory status, run and understand the output of free and vmstat. It is worth noting that the value of “cached” refers to the amount of memory the Linux kernel uses to cache files, regardless of free memory.

• Java system debugging is a different matter. One technique that can be used for debugging on An Oracle JVM or any other JVM is that you can run kill -3 < PID > while a full stack trace and heap overview (including GC details) are saved to standard error or log files. JPS, jSTAT, jStack, and JMAP in the JDK are useful. SJK Tools is more advanced.

• MTR is used to track routes and to determine network problems.

• Use NCdu to check disk usage, which is much more time efficient than regular commands such as du -sh *.

• Find the socket connection or process that is using bandwidth, using iftop or nethogs.

• The AB tool (which comes with Apache) is a simple and crude way to check web server performance. For more complex load tests, use siege.

• Wireshark, Tshark, and ngrep can be used for complex network debugging.

• Learn about Strace and LTrace. These two tools are very useful when your program fails, hangs, or even crashes, and you don’t know why or you want to get an overall idea of performance. Note the profile parameter (-c) and the parameter (-p) attached to a running process.

• Learn about using LDD to examine shared libraries. But never run on untrusted files.

• Learn how to use GDB to connect to a running process and get its stack trail.

• Learn to use /proc. It can be surprisingly effective at debugging emerging problems. Such as: /proc/cpuinfo, /proc/meminfo, /proc/cmdline, /proc/xxx/cwd, /proc/xxx/exe, /proc/xxx/fd, /proc/xxx/smaps (where XXX indicates the process ID or PID).

• SAR is very useful when debugging problems that have occurred before. It shows historical data about CPU, memory, network, etc.

• For deeper system analysis and performance analysis, see STAP (SystemTap), PerF, and sysDIG.

• To see which system you are currently using, use uname, uname -a (Unix/kernel information) or lsb_release -a (Linux distribution information).

• Try DMESG whenever something is working happily (perhaps hardware or driver issues).

• If you delete a file, but no release the disk space is expected by du found, please check if the file is occupied by a process that lsof | grep does | grep “filename – of – my – big – file”

A single script

Some examples of command combinations:

• When you need to do set intersection, union, and difference operations on a text file,sort 和 uniqWould be a great help to you. For specific examples, please refer to the code behind the assumption herea 与 bTwo files with different contents. This approach is highly efficient and can be applied to both small files and large files/tmpYou may need it on a small root partition-TParameter, but actuallysortIs not constrained by memory size), see the previous article onLC_ALL 和 sort 的 -uThe parameter part.

      sort a b | uniq > c   # C is a and B
      sort a b | uniq -d > c   # C is a crossing b
      sort a b b | uniq -u > c   # c is a minus b
Copy the code

• Use grep. * (each line has a file name attached) or head-100 * (each file has a title) to read the contents of all files in the check directory. This is especially useful when checking a directory full of configuration files such as /sys, /proc, /etc.

• Calculates the sum of all the numbers in the third column of a text file (possibly three times faster and three times less code than equivalent Python code) :

      awk '{ x += $3 } END { print x }' myfile
Copy the code

• If you want to look at the size/date in the file tree, this might look like a recursive versionls -lBut more thanls -lREasier to understand:

      find . -type f -ls
Copy the code

• Suppose you have a text file similar to a Web server log file, and a certain value will only appear on certain lines, let’s say oneacct_idThe parameters are in the URI. If you want to figure out each of theseacct_idValue for how many requests, using the following code:

      egrep -o 'acct_id=[0-9]+' access.log | cut -d= -f2 | sort | uniq -c | sort -rn
Copy the code

• To continuously monitor file modification, you can use the watch, check the change of the files in a folder, for example, can use a watch – d – n 2 ‘ls – RTLH | tail’; Or to monitor the change of network Settings when troubleshooting WiFi Settings, you can use watchd-n 2 ifconfig.

• Run this function to pick up a random tip from this document (parse the Markdown file and extract the project) :

      function taocl() {
        curl -s https://raw.githubusercontent.com/jlevy/the-art-of-command-line/master/README-zh.md|
          pandoc -f markdown -t html |
          iconv -f 'utf-8' -t 'unicode' |
          xmlstarlet fo --html --dropdtd |
          xmlstarlet sel -t -v "(html/body/ul/li[count(p)>0])[$RANDOM mod last()+1]" |
          xmlstarlet unesc | fmt -80
      }
Copy the code

Unpopular but useful

• Expr: Evaluates an expression or regular match

• M4: simple macro processor

• Yes: Print the string multiple times

• CAL: Nice calendar

• Env: Execute a command (useful in script files)

• Printenv: Print environment variables (useful for debugging or when writing script files)

• Look: Looks for words or lines that start with a particular string

• Cut, paste, and JOIN: Data modification

• FMT: Formats paragraphs of text

• Pr: Formats the text into a page/column format

• Fold: Several lines in the text of a package

• Column: Formats the text into multiple aligned, fixed-width columns or tables

• Expand and unexpand: Converts between tabs and Spaces

• Nl: adds a line number

• Seq: Print numbers

• BC: Calculator

• Factor: Factorization factor

• GPG: encrypts and signs the file

• Toe: Terminfo entry list

• Nc: network debugging and data transmission

• Socat: Socket proxy, similar to Netcat

• Slurm: Visualizes network traffic

• Dd: data is transferred between files or devices

• File: Determines the file type

• Tree: Displays paths and files in the form of a tree, similar to the recursive LS

• Stat: indicates file information

• Time: The command is executed and the execution time is calculated

• Timeout: The command is executed within the specified period and the process is stopped after the specified period

• Lockfile: Enables files to be removed only by rm -f

• Logrotate: Switch, compress, and send log files

• Watch: Run the same command repeatedly, displaying the results and/or highlighting the changes

• When -changed: The command is executed when a file change is detected. See inotifywait and entr.

• Tac: indicates the reverse output file

• Shuf: A random selection of lines from the file

• Comm: Compares sorted files line by line

• Strings: extracts text from a binary file

• Tr: Converts letters

• Iconv or UCONV: Text encoding conversion

• Split and csplit: Split files

• Sponge: reading all the input before writing, after reading the file to the same file is written to be useful, for example, grep -v something some – file | sponge some – file

• Units: convert a unit of measurement to another kind of equivalent units (refer to the/usr/share/units/definitions. The units)

• Apg: Randomly generates passwords

• Xz: High proportion of file compression

• LDD: dynamic database information

• Nm: Extracts the symbols in the OBj file

• Ab or WRK: Web server performance analysis

• Strace: Debug system call

• MTR: Better network debugging tracking tool

• CSSH: A visual concurrent shell

• Rsync: Synchronizes files and folders over SSH or the local file system

• Wireshark and Tshark: tools for capturing packets and debugging networks

• Ngrep: grep of the network layer

• Host and DIG: DNS lookup

• Lsof: lists the tools used to open files in the current system and view port information

• Dstat: displays the system status

• Glances: High-level overview of multiple subsystems

• Iostat: indicates the disk usage status

• Mpstat: INDICATES the CPU usage status

• Vmstat: memory usage status

• Htop: an enhanced version of top

• Last: Login record

• W: View the login users

• Id: indicates the ID of a user or group

• SAR: historical data of the system

• Iftop or nethogs: Network utilization of sockets and processes

• Ss: Socket data

• Dmesg: Boot and system error messages

• Sysctl: Dynamically view and modify kernel running parameters while the kernel is running

• Hdparm: SATA/ATA disk changes and performance analysis

• LSBLK: Lists block device information: displays your disk and disk partition information in a tree

• LSHW, LSCPU, LSPCI, LSUSB, and DMIDecode: View hardware information, including CPU, BIOS, RAID, graphics card, and USB device

• Lsmod and Modinfo: Lists the kernel modules and displays their details

• Fortune, Ddate and SL: Well, a lot depends on whether you think steam trains and silly quotes are “useful.”

OS X only

Here are some tips that are limited to OS X.

• Use BREW (Homebrew) or Port (MacPorts) for package management. These can be used to install most of the above commands on OS X systems.

• Use PBCopy to copy the output of any command to the desktop application and PBPaste to paste the input.

• To use the Option key as the Alt key on an OS X terminal (for example, in the alt-b, Alt-f commands described above), go to Preferences -> Description Files -> Keyboard and select use Option as the Meta key.

• In the open or open – a/Applications/Whatever. App opens the file using a desktop application.

• Spotlight: Search for files with MDFind and list metadata with MDLS (EXIF information for photos, for example).

• Note that OS X is based on BSD UNIX, and many of the commands (such as ps, ls, tail, awk, sed) are subtly different from those in Linux (which is heavily influenced by System V-style UNIX and GNU tools). You can see these differences on the man page titled “BSD General Commands Manual”. In some cases GNU versions of commands may also be installed (for example gawk and gsed correspond to AWk and sed in GNU). If you’re writing cross-platform Bash scripts, avoid using these commands (consider Python or Perl, for example) or do careful testing.

• Use SW_vers to obtain OS X version information.

Windows only

Here are some tips for Windows only.

Get the Unix tools under Windows

• Cygwin can be installed to allow you to experience the power of Unix shells in Microsoft Windows. In that case, most of the content in this article will apply.

• On Windows 10, you can use Bash on Ubuntu on Windows, which provides a familiar Bash environment that includes a number of Unix command-line tools. The advantage is that it allows programs written on Linux to run on Windows, whereas programs written on Windows cannot run on the Bash command line.

• If you primarily want to use GNU developer tools (such as GCC) on Windows, consider MinGW and its MSYS package, which provides tools such as bash, GAWk, make, and grep. MSYS does not contain all the features that are comparable to Cygwin. MinGW will be particularly useful when making native Windows ports for Unix tools.

• Another option for achieving the look and feel of a Unix environment under Windows is Cash. Note that very few Unix commands and command lines are available in this environment.

Use Windows command line tools

• You can use WMIC to script and execute most Windows system administration tasks in a command line environment.

• Native command-line networking tools for Windows include ping, ipconfig, tracert, and netstat.

• You can use the Rundll32 command to implement many useful Windows tasks.

Cygwin skills

• Install additional Unix programs through Cygwin’s package manager.

• Use Mintty as your command line window.

• To access the Windows Clipboard, go to /dev/clipboard.

• Run cygstart to open a file by default.

• To access the Windows registry, use the RegTool.

• Note that the Windows drive path C:\ is represented by /cygdrive/ C in Cygwin, and the/in Cygwin stands for C:\ Cygwin in Windows. To convert Cygwin and Windows style paths, use cygpath. This is useful in scripts that need to call Windows programs.

• With WMIC, you’ll be able to perform most Windows system administration tasks from the command line and script them.

• Another way to get the Unix interface and experience under Windows is to use Cash. Note that this environment supports very few Unix commands and command-line arguments.

• Another way to get GNU developer tools on Windows, such as GCC, is to use MinGW and its MSYS package, which provides bash, gawk, make, grep, and more. However, MSYS offers less functionality than Cygwin. MinGW is very useful in creating native Windows ports for Unix tools.

More resources

• Awesome – Shell: a well-organized list of command-line tools and resources.
• Awesome – OSX-command-line: A more in-depth guide to the OS X command line.
• Strict mode: To write better script files.
• Shellcheck: a static shell script analysis tool, essentially a bash/sh/ZSH Lint.
• Filenames and Pathnames in Shell: details about how to properly handle Filenames in Shell scripts.
• Data Science at the Command Line: Some commands and tools for Data Science, from the book of the same name.

This post was republished by GitHub Joshua Levy

Photo by Fernando Hernandez on Unsplash