My zhihu replies that the question is about how to read the Go source code. Address. He waffled a little on his own.
The source code for Go is in the SRC/directory of the installation package. How to read its source code? Just look! With no one to teach me, I had to masturbate. Of course, this kind of content is not usually taught.
How lu?
Go source code, should be divided into a language-related part, and the official standard library. The ones related to language implementation are definitely the hardest and not so easy to understand. Can see the main standard library first, the other can be roughly understood first.
First sweep the whole source directory, roughly see what modules are involved, and then pick your favorite part for further study and research. It is recommended that each directory simply write hello World, so that the understanding will be deeper. If you can’t even write Hello World, there is no need to study the source code of this module for the time being. After all, the use of packages is not only language-related, but also scenario-based and implementation principles that need to be learned.
Once you’re familiar with the package, you can read the source code, but it’s best not to get too detailed at this point and try to understand the design ideas involved, the overall flow. Source code reading can be aided by drawing UML to help understand both vertically and horizontally. When it comes to code design, the easiest thing to think about is to write sequentially. But as projects get bigger, there are more and more abstract modules, abstracting out interfaces and concrete implementations that may contain combinations of other types. Understanding these relationships is helpful in understanding the source code implementation.
If you can successfully go through the first two steps, the following source code reading is relatively easy. And Go is characterized by simplicity and ease of reading, with little grammatical sugar. Of course, for more complex packages, you need to know the underlying principles, such as the NET/HTTP package. You need to be familiar with the HTTP protocol before you can explore the source code implementation in detail.
May be my idle egg pain, ready to try to start from the first step, the overall Go source code contains modules, nothing when the update a bit into. I feel like I have a lot more material for my Golang Journey column.
My environment is Go 1.11. For each module, I will post some of the articles I have read below. Since this is only a cursory reading, there is no guarantee that every article I have read will be a good one.
Supplement:
At 01:13 am on August 8, 2019, IT took about two weeks to finish a simple version. Overall feeling, there are still many places to understand not enough, I hope that the back can be said in front of the idea, according to the package of source code anatomy step by step.
archive
Contains archiving related content, which involves two packages, tar and ZIP.
Archive /tar. You can use the tar command in Linux to archive files. If you want to compress on an archive basis, you also use related packages under COMPRESS. One caveat is to understand the difference between archiving and compression.
Related reading:
Birdbrother files and file system compression and packaging archive/tar package compression and decompression
Archive /zip, a package related to the operation of compressed files in zip format, which is used in a similar way to tar. While looking for information related to zip packages, we learned that the author of ZIP died at the age of 37, and that all files using ZIP compression in the world have his name “PK” at the beginning of the file, and we can identify whether a file is ZIP by this method.
Related reading:
Archive /zip To achieve compression and decompression zip Baidu Encyclopedia
bufio
The IO.Reader or IO.Writer function is wrapped to create a new Reader or Writer instance, and these newly created instances provide buffering capability. It is very simple to reach a specified buffer size, trigger a write or read operation, and Flush if not reached.
Related reading:
Introduction to Bufio package in Golang in-depth Introduction to Bufio.Scanner in Golang Bufio – Cache IO
builtin
Go language built-in type, function, variable, constant declaration. For the time being, there’s nothing more to read and you should read it in conjunction with Go’s internal implementation.
bytes
Functions for byte slice operations. Since []byte can also be used to represent string, its functions and methods are similar to strings, such as Join, Split, Trim, and Contains.
Related reading:
Go language learning -bytes Package Go Walkthrough: bytes+ + strings
cmd
The implementation code of the Go command tool set, such as Go, GOfmt, GODoc, pprof, etc., should be mainly related to the Go language implementation, relatively low-level. Each command can be studied for some time, especially the GO command, if you have a good understanding of the underlying principles of your computer.
A search on the Internet shows little information about it.
Related reading:
Command Go
compress
As mentioned earlier, the Archive package is related to archiving, while the relative COMPRESS package is mainly related to compression. The main implementation of several mainstream compression formats, such as Bzip2, Flate, gzip, LZW, zlib.
Compress /bzip2, the common.bz2-ending compressed file format basically works with this package, and should be used with tar.
Compress /gzip, the common. Gz end compression file format can be used in this package. Use this package together with tar.
Compress /flate, flate is the main compression algorithm used by zip. If you read the source code of archive/zip, you will find that the package is imported.
Compress /zlib, compress/ LZW are implemented by some compression algorithms. Because I do not have too deep research on compression algorithm, a temporary understanding of the general good, I hope not to introduce mistakes.
Related reading:
Go official website compress
container
As we know, Go has very few built-in data structures, just arrays, slicing, and mappings. In addition, there are also parts of the structure in the Container package: heap, list, and ring.
They are very simple to use, basically add, delete, change and check.
Related reading:
Container Indicates the data types of the container: Heap, List, and Ring
context
Before you can read this package, familiarize yourself with how Go’s concurrent code is written and how the Done Channel implements broadcasting to all goroutines. Go’s concurrency unit is called a Goroutine, but there is no father-son relationship between goroutines, and the context package was created for better concurrency control. It can safely transfer data between different Goroutines and manage timeouts.
Related reading:
Go translation through the context to achieve concurrent control depth decryption of the Go language context
crypto
Key encryption algorithms AES, DES, RSA, DSA, sha1, SHA256, rand, rand, rand, rand, rand, RAND, RAND, RAND Did not find a comprehensive introduction of the article, after all, more complex, if you want to see their source code, have to understand about the principle of each encryption algorithm, to break through one by one.
Related reading:
Crypto on Go’s official website
database
Encapsulate a set of general interface for database operation, database connection management, support connection pool function. In real use, we need to introduce the corresponding driver, to implement the operation of the specified type of database.
A simple example.
import (
"database/sql"
_ "github.com/go-sql-driver/mysql"
)
func main(a) {
db, err := sql.Open("mysql"."Username: password @ TCP (127.0.0.1:3306)/test")
iferr ! =nil {
log.Fatal(err)
}
defer db.Close()
}
Copy the codegithub.com/go-sql-driver/mysql is the MySQL driver provided. Specific query execution is done by calling the methods in the DB interface implemented by the driver.
Related reading:
Database/SQL-SQL/SQL-like Database operation interface Study notes about the database/ SQL package in Golang
debug
And debugging related, the specific content is more complex, I do not understand. There are several packages inside, such as Dwarf, Elf, Gosym, Macho, PE, Plan9obj.
Dwarf, for accessing DWARF information in executable files. What exactly is DWARF information? There’s a PDF on the website that explains DWARF, if you’re interested. It mainly provides necessary debugging information for the UNIX debugger, such as the file name and line number corresponding to the PC address, to facilitate source-level debugging.
Related reading:
Dwarf -2.0.0 debug information format dwarf, in case you need it
Elf, for accessing ELF type files. Elf, executable and connectable formats, often referred to as ELF formats, come in three types:
- A Relocatable object file, an.o file generated by the assembler
- Executable object files that can execute applications
- A Shared object file is a dynamic library file that can be Shared
Related reading:
The 101 of ELF Files on Linux: Understanding and Analysis
Gosym, which is used to access Go symbols and line information in binary files generated by the Go compiler, hasn’t been seen much yet. This is the second article in a medium series that introduces the implementation of the Debug debugger in Go.
Related reading:
Making debugger in Golang (part II)
Macho, used to access files in The Mach-o object format. To read the source code, you also need to understand what Mach-O is, which is short for Mach Object file format and is used for executables, object code, and kernel dumps.
Related reading:
Wikipedia -Mach -o Go package – debug/macho
PE, realize access to PE format files, PE is Windows portable executable file format.
Related reading:
WIKI – Portable Executable
Go package – debug/pe
Plan9obj, used to access plan9 object format files.
No introduction to Plan9Object found yet. Our main learning should focus on the ELF and GOSYm formats.
Related reading:
Go package – debug/plan9obj
encoding
It is mainly about the conversion operation of various data formats commonly used by us, or it can also be called codec, such as JSON, XML, CSV, BASE64, etc. The main modules are:
Encoding /json; encoding/json; encoding/json; map[string]interface{};
Encoding/XML, basically the same as encoding/json. But because XML is much more complex than JSON, it also involves some advanced uses, such as operations related to element attributes.
Encoding/CSV, parsing the CSV data format.
Encoding /binary, which can be used to process the lowest level of binary data streams, and convert []byte and integer data by size side.
Other data format operations such as HEX, GOB, base64, base32, GOb, PEM, ascii84, etc. are similar and should be tried if you are interested.
Related reading:
Encoding/XML; Golang; Golang; Golang; Golang; Golang; Golang Walkthrough: encoding package
errors
This is the main error-handling code for Go. Unfortunately, when I opened the source code, I found only a few lines of code. Mainly because Go’s error type is just an interface, its source code is very simple.
package errors
// New returns an error that formats as the given text.
func New(text string) error {
return &errorString{text}
}
// errorString is a trivial implementation of error.
type errorString struct {
s string
}
func (e *errorString) Error(a) string {
return e.s
}
Copy the codeGo provides the simplest implementation by default, just a few lines of code above. Awesome, amazing, lol. But that simplicity makes it easy to extend your own error. For example, some developers found Go’s error handling too easy and developed error packages containing call Stack Trace.
Related reading:
github.com/pkg/errors
error-handling-and-go
What I Don’t Like About Error Handling in Go
expvar
For example, after the HTTP service is added to expVAR, we can use /debug/vars to return these indicators. The data returned is in JSON format.
Its source code is small, only about 300 lines of code, the emphasis is on how it is used.
Related reading:
EXPVAR Monitoring apps with EXPvars and Go
flag
The package is used for parsing command line parameters, for example, grep -v grep. Obtain the parameter values after -v. Very common functions, if the pure implementation is more cumbersome.
Related reading:
Flag – Command line parameter parsing
fmt
As you can tell from the package name, FMT is mainly about formatting. Formatting of what? It’s basically string formatting, which is very similar to printf in C. Of course, in addition to implementing the usage of C, some go-specific implementations are provided.
Related reading:
Go Walkthrough: fmt
go
Seems to be the package used by the core tool.
hash
Hash packages define unified interfaces for different hash algorithms. Specific hash algorithms implement the lower level of direct hash, such as CRc32 and CRC64, that is, 32-bit cyclic redundancy check algorithm and 64-bit cyclic redundancy check algorithm. The MD5 hash algorithm also implements hash interfaces under crypto/ MD5.
Related reading:
Common hash functions FNV and MD5
html
The HTML package in the Go standard library is very simple. At a glance, it is mainly about the processing of HTML text, such as how to escape HTML code. If you want to support HTML parsing, go also provides a NET repository on Github, which has an HTML toolkit. Goquery is also implemented based on it.
The standard library’s HTML directory also contains Template, the TEMPLATE rendering tool for HTML, which can be combined with NET/HTTP and a database ORM package to start simple Web development.
Related reading:
Easy way to render HTML in Go
image
Go 2D image processing library, support to create 2D processing method functions, image creation, pixel, color Settings, and then draw. Supports PNG, JPEG, and GIF image formats.
Related reading:
Golang image/ Draw package golang image/ Draw package golang drawing technology
index
The directory is index, and there is only one package, index/suffixarray, called suffixarray. The specific algorithm has not been carefully studied, which is roughly to reduce the time complexity of substring query to
It is very simple to use, and an example is provided on the website.
// create index for some data
index := suffixarray.New(data)
// lookup byte slice s
offsets1 := index.Lookup(s, - 1) // the list of all indices where s occurs in data
offsets2 := index.Lookup(s, 3) // the list of at most 3 indices where s occurs in data
Copy the codeRelated reading:
Go package-index/Suffixarray suffix Array suffix algorithm
internal
Internal implementation, more complicated.
io
Go’s standard library provides a basic interface and implementation for IO primitives to aid byte stream reading. The main interfaces are IO.Reader and IO.Writer. The IO package provides interface implementations for common resources such as memory, files, and network connections to operate on.
Read the IO package source code, you will find that many interfaces are defined based on specific capabilities, the simplest are Reader (read), Writer (write), Closer (close), Seeker (offset), one interface one method, very flexible. Combined interface and ReaderWriter (read and write), ReadeCloser (read and write), WriteCloser (read and write) and ReadWriteCloser (read and write), etc. Overall understanding, we’ll get a better sense of how the Go interface is based on the duck model,
Related reading:
Basic IO interface Streaming IO in Go
log
Go log package, through logging can facilitate us to debug problems. The core source code for the log package is small, totaling just over 300 lines, with comments accounting for almost 100 lines. This is mainly because it provides very little functionality, only basic log formatting, as well as Print, Panic, and Fatal log printing functions. Even error levels are not provided. If you want to use it, you need to use some third-party packages. There is a “Go log library collection “article in the related reading, which I did not delve into.
Related reading:
Go log Indicates the Go log library
math
Some mathematical constants, such as PI (PI) and E (natural logarithm), are in it, as well as functions such as Round, Floor, Ceil, Max and Min in rounding. Complex mathematical operations, such as power operation, logarithm and trigonometric functions, must also exist. Other functions such as random numbers are also included. Open the Math source folder and find that there is a lot of assembly code in it. Mathematics is relatively low-level and requires high performance, so it is necessary to implement in assembly.
Math package, directly read the official documentation is good, generally read can be used, no business scenarios, specific principles need to understand, after all, we have learned math. If you look at assembly implementations, it gets complicated. You can study it if you’re interested.
Related reading:
Go website math
mime
To understand the use of MIME packages, you need to understand mime, which stands for Multipurpose Internet Mail Extension. The original design goal was to attach multimedia content to mail. MIME was later supported in HTML.
There are four main functions, AddExtensionType, TypeByExtension, FormatMediaType, and ParseMediaType. Both sets of functions appear to be interoperable with MediaType.
Related reading:
Go standard library learn MIME Go package – MIME
net
Network related, involving more content, a kind of unbearable feeling.
The underlying implementation of socket is in the NET package, mainly the implementation of some underlying protocols, such as connectionless IP, UDP, Unix (DGRAM), and connected TCP, Unix (STREAM) can be found in the NET package.
Application layer protocol, HTTP protocol implementation in NET/HTTP contains client server, RPC in NET/RPC, mail related NET /mail, NET/SMTP and so on. Net/URL is a function related to URL handling, such as URL string parsing, encoding, etc.
Related reading:
Golang Net Package Learning Notes Go official library RPC development Guide Go crawler essential HTTP request QuickStart Sending HTML emails using templates in Golang
os
The OS package mainly implements operating system-specific functions and is platform-independent. It is unix-style in design and uses Go error handling style. A variable of type error that will be returned if an error occurs. Such as Open, Stat, and other operation-related functions.
The goal of the OS package is to unify the functions of different operating systems. If you read the advanced UNIX environment program, you’ll see that the functions in the OS package are very similar to the UNIX system call functions.
In addition to OS packages, there are several packages in this directory, namely OS /exec, OS /signal, and OS /user, as follows:
OS /exec helps us implement the ability to easily execute external commands.
OS /signal, a UNIX-like function related to system signal processing. Linux supports 64 system signals.
OS /user: a library related to system users, which can be used to obtain information about the login user and the user group.
Related reading:
Use OS /exec to execute the command Go package-os
path
The path package implements some common functions related to path handling (by/separation), often used for paths such as file paths and URLS. Not suitable for Windows \ and disk path handling.
The main functions include Base, Clean, Dir, Ext, IsAbs, Join and so on. For example, Base can be used to obtain the last element of a path, Dir can obtain the path directory, Ext can obtain the file extension, IsAbs can determine whether it is an absolute path, Join can connect paths, etc.
Related reading:
Go package – path
plugin
The Plugin package is a Go 1.8 package that adds dynamic library loading capabilities to Go and currently only supports Linux and MacOS. However, the application of this package is not very convenient, and the environment in which the library files are generated and used has certain requirements.
Related reading:
How to evaluate the plugin package writing-modular- Go-programs-with-plugins calling- go-functions-from-otherlanguages added to Go standard library gosh-a-pluggable-command-shell-in-go
reflect
Function functions related to reflection, through reflection can achieve runtime dynamic creation, modification of variables, function method calls and other operations, to obtain the dynamic characteristics of the interpreted language. To read the reflection package source, focus on understanding the two components of a variable, namely type and value, around which reflection’s core operations are based. Reflect.valueof and Reflect.typeof are the two methods we use.
Related reading:
(2) Go package-reflect
regexp
Go’s re package for re processing. It’s available in almost every language. The methods involved can be roughly divided into several categories, respectively Compile Compile, Match Match, Find search, Replace.
Re source code implementation is really do not want to see. Feel regular is not completely clear, pull source a bit pit. Very big head.
Related reading:
Regexp Golang-reggex-Tutorial on Go
runtime
The Runtime is the implementation associated with the Go runtime, and it provides some functions to control Goroutine. The Go startup process, GC, goroutine scheduler, etc., are also implemented in Runtime. We also need to read the Runtime code to understand. In addition, cgo, Builtin package implementation is also in the Runtime.
Related reading:
Golang internals Go package-Runtime
sort
Defines an interface for sorting, and once a type implements an interface for sorting, you can implement sorting using functions in sort. By reading the source code, I found that the default sorting types are int, float64, and String. There is also a search file in sort, which is mainly an implementation of binary lookup for sorted content.
Sorting algorithm, as we all know, a lot of, such as insertion sort, heap sort and quick sort, sort packages have been realized, and need not we decided to use which kinds of algorithms, but what is on the basis of detailed data decided to use algorithm, and a sort is not necessarily as long as a kind of algorithm, and may be a combination of a variety of algorithm. You can learn how to do this by reading the quickSort function in the sort.go file.
Related reading:
The 3 ways to Sort in Go
strconv
For packages that convert strings to other types, the full name should be String convert. For example, integer and string conversion Itoa and Atoi, float and string conversion ParseFloat and FormatFloat, Boolean and string conversion ParseBool and FormatBool, and so on.
Related reading:
Use of strconv in Golang
strings
String manipulation functions have also been mentioned, because []byte can also be used to represent strings. Many of the strings functions have similar implementations in the bytes package, such as Join, Split, Trim, case conversion, and so on.
Related reading:
Strings-string operation
sync
Go recommends channels for concurrent synchronization control, but traditional mechanisms are also supported, such as locking mechanisms, condition variables, Waitgroups, atomic operations, and so on, which are provided by Sync. Atomic operations are in the sync/atomic package.
In addition, Sync also has a temporary object pool that is scalable and concurrency safe for object reuse.
Related reading:
Brief introduction to Golang Sync package
syscall
The system call, as you can tell by its name, is a very complicated package. System call is the interface to realize the application layer and the bottom layer of operations. The operations of different systems often have certain differences, especially between UNIX-like and Windows systems.
If you want to find use cases for Syscall, you can look at the source code of the NET, OS, and Time packages.
If you look at the source code, the current thinking is, we can just look at the Linux implementation, the architecture, and if we want to look at assembly, we can just look at the x86 architecture.
There is not much research, I dare not say anything.
Related reading:
Video Notes: Go and syscall Go package – syscall
testing
Test related implementations in Go, such as unit tests, benchmarks, and so on. The test approach recommended by Go takes a table-driven approach, where instead of writing a single use case for each case, you enumerate the inputs, expect the outputs, and then execute the functionality and compare the expected outputs to the actual outputs.
A simple test case.
func TestSum(t *testing.T) {
var sumTests = []struct {
a int
b int
expected int
}{
{1.1.2},
{2.1.3},
{3.2.5},
{4.3.7},
{5.5.10},
{6.8.14},
{7.13.20}},for _, tt := range sumTests {
actual := functions.Add(tt.a, tt.b)
ifactual ! = tt.expected { t.Errorf("Add(%d, %d) = %d; expected %d", tt.a, tt.b, actual, tt.expected)
}
}
}
Copy the codeRelated reading:
Unit test benchmarks Go package-testing
text
Mainly about text analysis analysis of some packages, but different from string processing, mainly involves lexical analysis scanner, template engine template, TAB processing tabwriter.
Text /scanner, is mainly to do lexical analysis, if you have read my column translation of several articles on lexical analysis, it will be easier to understand.
Text /template: used for text template processing. Text /template is more generic than HTML /template. To be familiar with it, you need to master some of its methods, such as Action, Argument, Pipeline, Variable, and Function.
Text /tabwriter: Text alignment based on tabs.
Related reading:
Text/Template A look at Go lexer/scanner Packages Go Template nesting best practices Package TabWriter
time
For date-time packages, Unix TIMESTAMP in Go is int64, representing a correspondingly extended time range. Other formats, such as sleep, time zone, timing control, etc., are supported in Go. There is a perverse rule in Go, that is, the date and time format string YYYY-MM-DD, for example, is 2006-01-02 in Go.
Related reading:
Golang Package time golang Package time golang Package time
unicode
Basic functions for Unicode encoding. If you read the source code, it provides functions to determine character types, such as control characters, letters, and so on, by sorting characters into different rangetables. The other two packages, Unicode/UTF8 and Unicode/UTf16, are used for conversions between Unicode (RUNe) and UTF8 (byte) and Unicode (Rune) and UTF16 (INT16).
Related reading:
Go Package unicode Unicode code points, UTF-8/16 encoding
unsafe
The Go language limits some usages that can cause a program to run erratically and can be checked for by the compiler. Of course, there are some problems that can’t be found at compile time, and Go gives a hint of optimization. However, with the methods presented in Unsafe, it’s possible to break through this layer entirely. As the package name implies, unsafe operations are more low-level. Some low-level packages call it, such as Runtime, OS, syscall, etc., which are closely related to the operating system. Unsafe is best avoided because using it doesn’t necessarily guarantee portability or future compatibility issues.
Related reading:
Go Bible – Low-level programming Go Package – Unsafe
vendor
The standard library relies on third-party packages, which are also officially developed by Go, including the following dependencies by default:
- golang_org/x/crypto
- golang_org/x/net
- golang_org/x/text
For example, encryption-related crypto packages are implemented using the methods in golang_org/x/crypto/curve25519.
In addition to the source code in the standard library, the official also provides a lot of other such as crypto, NET, text and other packages. For details, see the official Github address of Go.
