Writing in the front

Go build, go run, go install, go get… go build, go run, go install, go get… These commands can help us compile, run, install, and get code packages, but have you ever wondered how our source files are organized during go compilation? What happens when go install? And go get just to download packages that we depend on? What other useful commands do go have? This is a fairly basic article, but also serves as a quick guide to the GO command, designed to help us get a deeper understanding of GO.

Several concepts

  • Command source file: the source file that is declared as belonging to the main code package and contains the main function without parameter declaration and result declaration. Generally, there is only one command source file per project.
  • Library source file: a common source file that exists in a package without the characteristics of command source
  • Test source file: source file with the suffix _test.go and must contain a function whose name is prefixed by Test or Beanchmark.

Gopath is probably the most familiar thing for every GO developer. Gopath is our workspace. You can view our gopath using the Go env, such as gopath =”/Users/code/go”. The bin, SRC, and PKG folders are created under gopath.

  • Bin: The executable file after go install is stored in the bin directory. By default, GOBIN is empty, but when GOBIN is set, GOPATH/bin becomes meaningless, and the go install executable is placed in GOBIN by default.
Export GOBIN=/ TMP // Declare the GOBIN directory go install $GOPATH/ SRC /code // Install the code program/TMP/Code discovery program has been installed in GOBINCopy the code
  • SRC: The source files we wrote are in this folder.
  • PKG: the location where the library source files are stored after the dependency package is installed, that is, the archive file (at the end of.a), that is, the static library files generated after the program is compiled.

Common auxiliary commands

Go has some auxiliary commands that can help us better understand the execution process of go. The auxiliary commands are go run/build.. -x xx.go, -x is the auxiliary command. -x can have the following types:

  • -a: Force a rebuild of all involved code packages, including those in the standard library, even if they are up to date.
  • -n: Prints other commands used during the build, but does not actually execute them. -n allows you to watch the build process
  • -race: Used to detect data contention problems, such as map concurrent reads and writes…
  • -v: prints the code packages involved in the execution of the command, including the code packages we depend on, and can also see the code packages associated with the dependent code packages.
  • -work: is used to print the name of the temporary working directory that is generated and used during the execution of the command and is not deleted after the command is executed. If you do not add this tag, the temporary working directory will be deleted after the command is executed.
  • -x: Prints all the commands used in the execution of the command and executes them simultaneously.
  • -p n: Indicates the number of parallelisms (n) built. By default, the number of parallelisms is the same as the number of cpus.
  • -o: compiles the specified output file.

Minority auxiliary commands

  • -asmflags: This tag can be followed by other tags, such as -d, -i, -s, etc. The tags followed by these control the behavior of the Go compiler as it compiles assembler language files.
  • -buildmode: specifies the buildmode, such as -buildmode=default, which controls the compiler to generate static (.a files), dynamic (.so files) or/and executables (.exe files in Windows) after compilation.
  • -compiler: Specifies the name of the compiler currently in use, which can be GC or GCCGO. The GC compiler is the built-in editor of the Go language, while the GCCGO compiler is the Go language compiler provided by GCC.
  • -gccgoflags: Specifies a list of tags to be passed to the GCCGO compiler or linker.
  • -gcflags: Specifies a list of tags to be passed to the go tool compile command.
  • -installsuffix: To separate the current output directory from the default compiled output directory, use this tag. The value of this tag is used as a suffix for the parent directory name of the result file. In fact, if the -race flag is used, the flag will be appended automatically and its value will be race. If we used both the -race tag and -installsuffix, we would append _race to the value of the -installsuffix tag and use that as the actual suffix.
  • -ldflags: Specifies the list of tags to be passed to the go Tool link command.
  • -linkshared: used with -buildMode =shared. The latter causes non-main code packages that are the target of compilation to be merged into a dynamic link library file, while the former is linked on top of it.
  • pkgdir: You can specify a directory. The compiler loads only archive files of code packages from this directory and places archive files of code packages that might be generated by compilation in this directory.
  • -tags: Specifies a list of compilation labels (also known as compilation constraints) that need to be processed during actual compilation.
  • -toolexec: allows us to customize how to use some of Go’s built-in tools (vet, ASM, etc.) during compilation.
  • -mod: Use -mod [mode] mode, [mode] support readonly, release, vendor, when run the go build-mod =vendor command, the executable file will be generated and the project dependency package will be placed in the vendor directory of the main module.
  • -modcacherw: Keep the downloaded mod package in the module cache as read/write rather than read-only.
  • -trimpath: Delete the source path information from binary files. For example, when our program panic, we may print out the source path of the related error, through this can hide.

compile

go run

Let’s take a look at the build process with go run -n main.go

mkdir -p $WORK/b001/ cat >$WORK/b001/importcfg.link << 'EOF' # internal packagefile command-line-arguments=/Users/gopher/Library/Caches/go-build/f5/f58206d3722b7787f6341e5013f38f3887e44138ffbdafbb07b67da3 77632762-d packagefile code/utils=/Users/gopher/Library/Caches/go-build/89/8931b87411d98f32ea6222ad605f3367700be896828286782db3e51de73217ff-d packagefile fmt=/usr/local/go/pkg/darwin_amd64/fmt.a ... EOF mkdir -p $WORK/b001/exe/ cd . /usr/local/go/pkg/tool/darwin_amd64/link -o $WORK/b001/exe/main -importcfg $WORK/b001/importcfg.link -s -w -buildmode=exe -buildid=ttUV5epeZDG7q3yVlG2A/ek8iSJ8rjD-RiujnYKAd/hNomEMXuvdA3I1ks6XOE/ttUV5epeZDG7q3yVlG2A -extld=clang /Users/goper/Library/Caches/go-build/f5/f58206d3722b7787f6341e5013f38f3887e44138ffbdafbb07b67da377632762-d $WORK/b001/exe/mainCopy the code
  1. Create a temporary directory: $WORK/b001/
  2. Find dependency package information: $WORK/b001/importcfg.link
  3. Create an EXE directory: $WORK/b001/exe/
  4. Use the go tool tool to link the library file (importcfg.link) to generate the executable file $WORK/b001/exe/main

Go run-work main.go, let’s take a look at the temporary folder:

go run -work main.go
WORK=/var/folders/s4/2cpbmp4s1_j4y3zv2s08m9q40000gn/T/go-build281107053
Copy the code

Cut to a temporary directory:

├─ b096 │ ├─ exe │ ├─ main ├─ import.linkCopy the code
  • Exe: our executable file.
  • Importcfg. link: the package our program depends on. Link the packages through the linker.

By default, temporary folders are deleted after the go run command is executed.

go build

Go build is used to compile our program. By default, the compiled file is stored in the current folder. If you specify -o, you can move it to the specified file. Let’s look at the build process by going build-n main.go:

# # command-line-arguments # mkdir -p $WORK/b001/ cat >$WORK/b001/_gomod_.go << 'EOF' # internal package main import _ "unsafe" //go:linkname __debug_modinfo__ runtime.modinfo var __debug_modinfo__ = "0w\xaf\f\x92t\b\x02A\xe1\xc1\a\xe6\xd6\x18\xe6path\tcommand-line-arguments\nmod\tcode\t(devel)\t\n\xf92C1\x86\x18 r\x00\x82B\x10A\x16\xd8\xf2" EOF cat >$WORK/b001/importcfg << 'EOF' # internal # import config packagefile code/utils=/Users/sunkang/Library/Caches/go-build/89/8931b87411d98f32ea6222ad605f3367700be896828286782db3e51de73217ff-d packagefile fmt=/usr/local/go/pkg/darwin_amd64/fmt.a packagefile runtime=/Users/sunkang/Library/Caches/go-build/b4/b44856e241a6bb3baf596eb19e4566e956a490ef403c1ed31ba8f014542fcf81-d EOF  cd /Users/gopher/go/src/code /usr/local/go/pkg/tool/darwin_amd64/compile -o $WORK/b001/_pkg_.a -trimpath "$WORK/b001=>" - p = go1.15 main - lang - complete - buildid Fjkl2yr7MirhGqbO0lrl/Fjkl2yr7MirhGqbO0lrl - goversion go1.15.3 - D _/Users/sunkang/go/src/code -importcfg $WORK/b001/importcfg -pack -c=4 ./main.go $WORK/b001/_gomod_.go /usr/local/go/pkg/tool/darwin_amd64/buildid -w $WORK/b001/_pkg_.a # internal cat >$WORK/b001/importcfg.link << 'EOF' # internal packagefile command-line-arguments=$WORK/b001/_pkg_.a packagefile code/utils=/Users/sunkang/Library/Caches/go-build/89/8931b87411d98f32ea6222ad605f3367700be896828286782db3e51de73217ff-d packagefile fmt=/usr/local/go/pkg/darwin_amd64/fmt.a ... EOF mkdir -p $WORK/b001/exe/ cd . /usr/local/go/pkg/tool/darwin_amd64/link -o $WORK/b001/exe/a.out -importcfg $WORK/b001/importcfg.link -buildmode=exe -buildid=s0BcVGdGaAeuHGFL7teJ/Fjkl2yr7MirhGqbO0lrl/Fjkl2yr7MirhGqbO0lrl/s0BcVGdGaAeuHGFL7teJ -extld=clang $WORK/b001/_pkg_.a /usr/local/go/pkg/tool/darwin_amd64/buildid -w $WORK/b001/exe/a.out # internal mv $WORK/b001/exe/a.out mainCopy the code

The overall process is similar to go Run, except that after compile and link, the generated executable files are moved to the current folder instead of dying with the temporary folder.

go install

The go install command is used to compile and install specified packages and their dependencies. The go install command is used to install the specified packages before they are compiled and installed. Unlike Go Build, Go Install places the compiled installation package in a specified folder. The installed code package will be stored in the PKG directory of the current workspace, that is, the archive file of. A. If we do not set GOBIN, the installed command source file will be stored in the bin directory of the current workspace, when we set GOBIN, it will be placed in GOBIN. Suppose the project now looks like this:

├── main.go │ ├── go │ ├── goCopy the code

Main. go is our entry file, the command source file, and utils is our dependency, the library source file. Go install -n main.go

mkdir -p $WORK/b001/
cat >$WORK/b001/importcfg.link << 'EOF' # internal
...
EOF
mkdir -p $WORK/b001/exe/
cd .
/usr/local/go/pkg/tool/darwin_amd64/link -o $WORK/b001/exe/a.out -importcfg $WORK/b001/importcfg.link -buildmode=exe -buildid=yfU8nbngCa6KbgUlJLKa/Fjkl2yr7MirhGqbO0lrl/khh18opCAdXA909bR95q/yfU8nbngCa6KbgUlJLKa -extld=clang /Users/sunkang/Library/Caches/go-build/ed/ed5868e69c99c66b8bf4b399e989ea410063b44143b546fd3f4a98f758d73a47-d
/usr/local/go/pkg/tool/darwin_amd64/buildid -w $WORK/b001/exe/a.out # internal
mkdir -p /Users/gopher/go/bin/
mv $WORK/b001/exe/a.out /Users/gopher/go/bin/main
Copy the code

You can see that the last line differs from the Go build in that it moves the executable into GOPATH/bin. When we run go install -n in the dependency package utils:

mkdir -p $WORK/b001/
mkdir -p /Users/gopher/go/pkg/darwin_amd64/code/
mv /Users/gopher/Library/Caches/go-build/89/8931b87411d98f32ea6222ad605f3367700be896828286782db3e51de73217ff-d /Users/gopher/go/pkg/darwin_amd64/code/utils.a
Copy the code

You can see that dependency packages are installed under GOPATH/ PKG and named as archive files ending in. A. Darwin_amd64 is an assembly of $GOOS_$GOARCH, the operating system and processor architecture, and code is our project name.

/ Users/gopher/go/PKG/darwin_amd64 / code └ ─ ─ utils. ACopy the code

So the go install process looks like this:

go get

The go get command is used to download and install a dependency package. The go get download source file is placed in the first workspace in GOPATH. Since Go began to support the Go mod in 1.11, when we enable the Go mod, the code obtained through go get will be downloaded to GOPAT/ PKG /mod. Optional parameters are supported after Go get

  • -d: Causes the command program to perform only the download action, but not the install action.
  • -u: Enables commands to use the network to update existing code packages and their dependencies. By default, this command only downloads code packages that do not exist locally from the network, and does not update existing code packages.
  • -t: Causes the command program to simultaneously download and install the code packages that are dependent on the test source files in the specified code package.
  • -insecure: Allows a command program to download a specified code package using an insecure scheme such as HTTP. If you are using a code repository (such as the company’s internal Gitlab) that does not have HTTPS support, you can add this tag. Use it when you are sure it is safe.
  • -v: Displays the executed commands.

go mod

Go has supported the Go mod mode since 1.11, and now I believe that most of you are using the Go mod. Compared to the advantage of vendor, after using the Go mod, your code can exist anywhere.

  • GO111MODULE=on Enables the go mod mode using this environment variable
  • GOPROXY=”goproxy. IO “, it is recommended that you use this proxy to download third-party dependencies of go.
  • GOPRIVE=”git.xx.xx”, prive is a private repository, some such as the company’s own internal repository, you can configure the domain name of the private repository to get internal code

Go mod Related commands

  1. Go mod init Module nameInitialize a GO project
  2. go mod downloadDownload Modules to the local cache, i.e$GOPATH/pkg/mod/cache/download,go mod download -jsonYou can output information in JSON format:
//go mod download-json {"Path": "github.com/go-basic/uuid", "Version": "v1.0.0", "Info": "/ Users/gopher/go/pkg/mod/cache/download/github.com/go-basic/uuid/@v/v1.0.0.info GoMod", "" : "/ Users/gopher/go/pkg/mod/cache/download/github.com/go-basic/uuid/@v/v1.0.0.mod", "Zip" : "/ Users/gopher/go/pkg/mod/cache/download/github.com/go-basic/uuid/@v/v1.0.0.zip", "Dir" : "/ Users/gopher/go/pkg/mod/github.com/go-basic/[email protected]", "Sum" : "h1:Faqtetcr8uwOzR2qp8RSpkahQiv4+BnJhrpuXPOo63M=", "GoModSum": "h1:yVtVnsXcmaLc9F4Zw7hTV7R0+vtuQw00mdXi+F6tqco=" }Copy the code
  1. go mod edit

The options for editing the go.mod file are -json, -replace… For example, if you want to modify a package, you can go mod edit-replace =old[@v]=new[@v]. This command is often used to edit the go.mod file.

  1. go mod graph

Prints dependent packages in text mode, as my go.mod is

Module code go 1.15 require (github.com/gin-gonic/gin v1.7.4 // indirect github.com/go-basic/uuid v1.0.0 // indirect)Copy the code

Go Mod Graph is executed

//go mod graph
code github.com/gin-gonic/[email protected]
code github.com/go-basic/[email protected]
github.com/gin-gonic/[email protected] github.com/gin-contrib/[email protected]
github.com/gin-gonic/[email protected] github.com/go-playground/validator/[email protected]
github.com/gin-gonic/[email protected] github.com/golang/[email protected]
github.com/gin-gonic/[email protected] github.com/json-iterator/[email protected]
github.com/gin-gonic/[email protected] github.com/mattn/[email protected]
github.com/gin-gonic/[email protected] 
...
golang.org/x/[email protected] golang.org/x/[email protected]
Copy the code

Go get -u github.com/PaulXu-cn/go-mod-graph-chart/gmchart to check the dependencies:

  1. go mod tidy

Add missing or remove unwanted modules. Currently I have a uUID package in my go.mod, but my code doesn’t reference it.

Module code go 1.15 require github.com/go-basic/uuid v1.0.0 // indirectCopy the code

Go mod Tidy:

The module code go 1.15Copy the code

You’ll find yourself removing my unwanted bag.

6. Go mod Verify to verify that the dependency is correct. 7. Go mod why Explain why packages and modules are needed. For example, run: go mod why github.com/go-basic/uuid and output:

# github.com/go-basic/uuid
code/utils
github.com/go-basic/uuid
Copy the code

My understanding is that code/utils is a useful package to github.com/go-basic/uuid.

go.sum

The go.sum file does two things:

  1. Checksum: Prevents packet tampering
  2. Record the update record of the package

When we go get a package, will be downloaded to the local $GOPATH/PKG/mod/cache/download, download down. There will be a named vx X.X.Z IP package, as well as vx. X.X.Z iphash file, Vx. X.X.Z iphash contents is vx. X.X.Z IP after the hash value, such as: h1: jwqTeEM3x6L9xDXrCxN0Hbg7vdGfPBOTIkr0 + / LYZDA = % uuid package as examples: When we go get github.com/go-basic/uuid, in addition to adding a require command to go.mod, we also write two records in go.sum:

Github.com/go-basic/uuid v1.0.0 h1: Faqtetcr8uwOzR2qp8RSpkahQiv4 + BnJhrpuXPOo63M = github.com/go-basic/uuid v1.0.0 / go mod h1:yVtVnsXcmaLc9F4Zw7hTV7R0+vtuQw00mdXi+F6tqco=Copy the code

The first hash is the hash value of the zip package we mentioned above, and the second hash is the hash value of the go.mod if our dependency package has a go.mod. In order to ensure the reliability of the dependency package, go will check the hash value of the dependency package through the server pointed to by go environment variable GOSUMDB=”sum.golang.org” after downloading. If the queried hash value is different from the local hash value, the downward execution is rejected and go.sum is not updated.

go clean

  • -n: Print out the cleanup command that needs to be executed, but do not execute it, so it is easy to see how the underlying operation is performed.
  • -i: Clears associated installed packages and runnable files. We can do this by combining-nTake a look at-iHow it works:
//go clean -i -n CD /Users/gopher/go/job // Current project rm -f job job.exe job.test job.test.exe main main.exe rm -f /Users/gopher/go/bin/jobCopy the code

Exe,. Test, and so on. Finally, try to delete $GOPATH/bin/job.

  • -r: loop to clear imported packages in import:
// go clean -r -n cd /Users/gopher/go/job rm -f job job.exe job.test job.test.exe main main.exe cd /usr/local/go/src/fmt  rm -f fmt.test fmt.test.exe cd /usr/local/go/src/errors rm -f errors.test errors.test.exe cd /usr/local/go/src/internal/reflectlite rm -f reflectlite.test reflectlite.test.exe ....Copy the code

The job project depends on a number of packages, which also perform the deletion of some compiled files in the current directory.

  • -x: Print out the detailed command to execute, which is-nThe execution version of the print.
  • -cache: Delete allgo buildThe cache of commands.

The go build process generates caches that are stored in the go environment variable GOCACHE. -cache removes the associated caches:

//go clean -n -cache
rm -r /Users/gopher/Library/Caches/go-build/00 
/Users/gopher/Library/Caches/go-build/01
/Users/gopher/Library/Caches/go-build/02 
/Users/gopher/Library/Caches/go-build/03 
/Users/gopher/Library/Caches/go-build/04 
...
rm -r /Users/gopher/Library/Caches/go-build/ff
Copy the code
  • -testcache: Deletes all test results of the current package.

When we use go test. To run a test case under a certain path, we compile and test each test file under the path, and cache the test results to avoid unnecessary test repeats. When the cache is successful, we will find cached on the second run of test.

Go test.ok job 0.431s Go test.OK job (cached)Copy the code

Go clean -testcache removes the corresponding testcache.

Go clean -testcache go test. ok job 0.459sCopy the code
  • -modcache: Deletes cached files in go.mod mode.

When we start go.mod mode to organize our go code, the downloaded dependencies are placed in $GOPATH/ PKG /mod. Going clean -modcache removes all files in $GOPATH/ PKG /mod.

//go clean -n -modcache
rm -rf /Users/gopher/go/pkg/mod
Copy the code

conclusion

The purpose of this article is mainly because of my lack of knowledge. I have used GO for two years, but I still have a vague sense of some go commands and the compilation process of GO. For example, the go.mod that has been used in the project, how does go.mod manage the code package that we rely on, and what is go.sum? Why do you need go.sum? What happens in the go build process? On the other hand, this article also lists the basic daily development of enough commands, so that when you need to find commands do not have to go to the Internet Baidu, can be used as a quick reference manual. Maybe our daily tasks don’t require us to know the underlying knowledge, but being curious is one way to get ahead as a programmer. Principles, the bottom of these very boring things, if you can chew down, you will find that many problems have been figured out. For example, when practicing martial arts, those who are high in martial arts will find that their inner kungfu mind method is very strong. When their inner kungfu mind method is strong, they will learn any martial arts quickly.