Using Go Modules by Tyler Bui-Palsulich, Eno Compton

introduce

Go 1.11 and 1.12 include preliminary modules support, and Go’s new version management system is used to rely on version information descriptions and easier administration. This blog post is a basic tutorial on getting started with Modules. A future article will cover publishing modules that others can use.

Modules is a collection of Go packages stored in the top-level directory in a file called go.mod. The go.mod file defines the module’s path, which is referred to as the root directory; The file also contains other package dependencies that can be built properly. Each dependency requirement is also labeled as a module path and labeled according to semantic versioning.

Starting with Go 1.11, the Go command line automatically enables modules if the go.mod file exists in the current or upper directory and outside the $GOPATH/ SRC directory. When the directory is in $GOPATH/ SRC, the go command still uses GOPATH mode for compatibility reasons, even if the go.mod file is present. Refer to the Go command line documentation). Starting with Go 1.13, Modules will be turned on by default during all development.

This blog will demonstrate a series of common operations to develop Go code using Modules:

  • Create a module
  • Add the dependent
  • Upgrade depend on
  • Add a new major version of a dependency
  • Upgrade a dependency to the new major version
  • Remove useless dependencies

Create a new module

Let’s start by creating a new module.

Create a new empty folder outside $GOPATH/ SRC, use CD to switch to this directory, and create a new source file called hello.go:

package hello

func Hello() string {
	return "Hello, world."
}
Copy the code

Let’s also create a test file called hello_test.go:

package hello import "testing" func TestHello(t *testing.T) { want := "Hello, world." if got := Hello(); got ! = want { t.Errorf("Hello() = %q, want %q", got, want) } }Copy the code

Now, this directory contains a package, but it is not a module because there are no go.mod files. If you created the file in /home/gopher/hello directory, we can see the result when we run the go test command:

$go test PASS OK _/home/gopher/hello 0.020s $Copy the code

The last line represents the summary information for the entire package test. Because we are now outside of $GOPATH and do not belong to any module, the go command does not know the reference path to the current directory, so it generates a pseudo-path using the current folder name.

Now let’s use the go mod init command to set the current directory as the module’s root directory, and try the go test command again:

$ go mod init example.com/hello go: creating new go.mod: Module example.com/hello $go test PASS ok example.com/hello 0.020s $Copy the code

Congratulations to you! You have written and tested your first module.

The go mod init command creates a go.mod file:

$cat go.mod module example.com/hello go 1.12 $Copy the code

The go.mod file only appears in the root directory of the module. The package reference path in a subdirectory takes the form of the module reference path plus the subdirectory path. For example, if we create a subdirectory named world, we don’t need to use the go mod init command in the subdirectory. The package is automatically identified as part of example.com/hello with a reference path to example.com/hello/world.

Add a dependency

The main motivation for the Go Modules feature is to improve the experience of using another developer’s code (or adding a dependency).

Let’s update Hello. go to introduce rsc. IO /quote and use it to implement Hello:

package hello

import "rsc.io/quote"

func Hello() string {
	return quote.Hello()
}
Copy the code

Now let’s run the test again:

IO /quote v1.5.2 GO: Downloading Rsc. IO /quote v1.5.2 GO: Complete rsc. IO /quote v1.5.2 GO: IO /sampler V1.3.0 go: Finding golang.org/x/text v0.0.0-20170915032832-14c0d48eAD0c go: IO/Sampler V1.3.0 GO: Downloading RSC. IO/Sampler V1.3.0 GO: Downloading golang.org/x/text v0.0.0-20170915032832-14 c0d48ead0c go: Fully achieving golang.org/x/text v0.0.0-20170915032832-14c0D48eAD0C PASS OK example.com/hello 0.023s $Copy the code

The go command automatically handles the versions of dependencies specified in go.mod. When the package import points to a module that is not in the go.mod file, the go command automatically searches for the module and adds the latest version to the go.mod file. (” Latest version “means the latest non-pre-released stable version of the label.) In our example, go test resolves the new reference path rsc. IO /quote as the corresponding module, version V1.5.2. It also downloads the two dependencies used in rsc. IO /quote, named rsc. IO /sampler and golang.org/x/text. However, only dependencies that are directly used are recorded in the go.mod file.

$cat go.mod module example.com/hello go 1.12 require rsc. IO /quote v1.5.2 $Copy the code

This is not repeated the second time you run the go test command because go.mod is up to date and all downloaded modules are cached locally (stored in the $GOPATH/ PKG /mod directory) :

$go test PASS OK example.com/hello 0.020s $Copy the code

It is worth noting that while the go command is easy and convenient to add new dependencies, it is not without cost. There may be problems with correctness, security, and improper authorization when your modules recursively rely on new dependencies. For more reflection, see Russ Cox’s blog on our software dependency issues.

As we saw above, adding a new dependency often leads to new indirect dependencies. The go list -m all command lists all current dependencies and their dependencies

$go list -m all example.com/hello golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c rsc. IO /quote v1.5.2 RSC. IO/sampler v1.3.0 $Copy the code

In the go List output, the current module, or main module, is usually the first line, followed by dependencies sorted by dependency path.

Version v0.0.0-20170915032832-14c0d48eAD0c of golang.org/x/text is an example of a pseudo-version, the version syntax of the go command used to mark unlabeled commits.

Meanwhile, the go.mod and go commands maintain a file named go.sum that contains the desired encrypted hash for the specified module version:

$cat go.sum golang.org/x/text v0.0.0-20170915032832-14c0d48eAD0c h1:qgOY6WgZO... $cat go.sum golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c h1:qgOY6WgZO... Golang.org/x/text v0.0.0 c0d48ead0c/go - 20170915032832-14 mod h1: Nq... RSC. IO/quote v1.5.2 h1: w5fcysjrx7yqtD/aO + QwRjYZOKnaM9Uh2b40tElTs3... RSC. IO/quote v1.5.2. / go mod h1: LzX7hefJvL54yjefDEDHNONDjII0t9xZLPX... RSC. IO/sampler v1.3.0 h1:7 uvkifmebqhfdjd + gZwtXXI + RODJ2Wc4O7MPEh/Q... RSC. IO/sampler v1.3.0 / go mod h1: T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9... $Copy the code

The go command uses the go.sum file to ensure that subsequent module downloads will have the same file content as the first download, ensuring that your project dependencies will not have unexpected malicious modifications, unexpected problems, and other problems. Both go.mod and go.sum need to be put into version management.

Upgrade depend on

In Go Modules, versions are marked with semantic version tags. The semantic version consists of three parts: major version, minor version, and revised version. For example, v0.1.2, major version 0, minor version 1, and revision 2.

Let’s upgrade the minor version in this section and try to upgrade the major version in the next section.

From the go list -m all output, we can see that we are using an untagged version of golang.org/x/text. Let’s upgrade this version to the latest version and then test if everything works properly:

$go get golang.org/x/text go: Finding golang.org/x/text v0.3.0 go: Downloading golang.org/x/text v0.3.0 go: Fully realize golang.org/x/text v0.3.0 $GO test PASS OK example.com/hello 0.013s $Copy the code

Wow, everything works just fine.

Let’s take a look at the go list -m all and go.mod files:

$ go list -m all
example.com/hello
golang.org/x/text v0.3.0
rsc.io/quote v1.5.2
rsc.io/sampler v1.3.0
$ cat go.mod
module example.com/hello

go 1.12

require (
	golang.org/x/text v0.3.0 // indirect
	rsc.io/quote v1.5.2
)
$
Copy the code

The golang.org/x/text file has been updated to the latest label version v0.3.0. Records in the go.mod file have also been updated to V0.3.0. The indirect annotation marks that a dependency is not used directly by the current module, but is only referenced indirectly in other dependencies. You can see more about this by going to Help Modules.

Now, we can try to upgrade the smaller version of rsc. IO /sampler, and again, we execute the test after using the go get command:

IO/Sampler GO: Downloading Rsc. IO/Sampler v1.99.99 GO: Downloading rsc. IO/Sampler v1.99.99 Go: downloading rsc. IO/Sampler v1.99.99 Rsc. IO /sampler V1.99.99 $GO Test -- FAIL: TestHello (0.00s) Hello_test. Go :8: Hello() = "99 bottles of beer on the wall, 99 bottles of beer, ..." , want "Hello, world." FAIL exit status 1 FAIL example.com/hello 0.014s $Copy the code

Oh, no! Testing shows that the latest version of rSC. IO /sampler is not compatible with the way we use it, so let’s take a look at all the available tag versions of this module:

IO /sampler Rsc. IO /sampler v1.0.0 v1.2.0 v1.2.1 v1.3.0 v1.3.1 v1.99.99 $go list-m -versions rsc. IO /sampler v1.0.0 v1.2.0 v1.2.1 v1.3.0 v1.3.1 v1.99.99 $Copy the code

The version we are using is V1.3.0, it seems that V1.99.99 is obviously not available, maybe we can use v1.3.1:

$go get Rsc. IO /[email protected] go: Finding Rsc. IO /sampler v1.3.1 GO: Downloading rsc. IO /sampler v1.3.1 go: Rsc. IO/Sampler V1.3.1 $GO test PASS OK example.com/hello 0.022 S $Copy the code

Notice that in the go get command we use ‘@v1.3.1 as an argument. Normally, go get can specify the version to be used for parameter markers. The default is @latest, which will attempt to use the latest version.

Add a major version of the dependency

Let’s add a new function to our package: Func Wisdom returns a Concurrency maxim for Go, which is implemented in the Rsc. IO /quote/v3 module by calling Quote.concurrency.

Now, let’s update Hello. go to add a new function:

package hello

import (
	"rsc.io/quote"
	quoteV3 "rsc.io/quote/v3"
)

func Hello() string {
	return quote.Hello()
}

func Proverb() string {
	return quoteV3.Concurrency()
}
Copy the code

Now, let’s add the corresponding test:

func TestProverb(t *testing.T) { want := "Concurrency is not parallelism." if got := Proverb(); got ! = want { t.Errorf("Proverb() = %q, want %q", got, want) } }Copy the code

Now let’s test it out:

IO /quote/v3 v3.1.0 go: Downloading rsc. IO /quote/v3 v3.1.0 go: downloading rsc. IO /quote/v3 v3.1.0 go: Rsc. IO /quote/v3 V3.1.0 PASS OK example.com/hello 0.024s $Copy the code

IO/rsc. IO/rsc. IO /quote/v3

$ go list -m rsc.io/q... Rsc. IO /quote v1.5.2 rsc. IO /quote/v3 v3.1.0 $Copy the code

Each different major version (v1, v2, and so on) can use a different reference path: the path ends with the major version. IO /quote v3 version of rsc. IO /quote is no longer rsc. IO /quote but rsc. IO /quote/v3. This is called semantically referenced versioning and allows incompatible packages (usually different major versions) to have different paths. Among the differences, v1.6.0 of rsc. IO /quote requires forward compatibility with V1.5.2 so that the rsc. IO /quote name can be reused. (In the previous section, v1.99.99 of rsc. IO/Sampler should be compatible with v1.3.0 of Rsc. IO /sampler, but bugs or other reasons can cause this problem.)

The go command allows a build to contain up to one version of a specified module path, meaning that each major version can contain up to one version: one rsc. IO /quote, one Rsc. IO /quote/v2, and one Rsc. IO /quote/v3.

This prescribes a clear rule for module authors on a single module path: a program can be built under both V1.5.2 and V1.6.0 versions of RSC. IO /quote. Also, allowing different main versions of modules (because there are different paths) allows consumers of modules to incrementally upgrade to the new main version. For this example, we wanted to use the quote.Concurrency file in RSC /quote/ V3 V3.1.0, but this capability would help us if we didn’t combine our RSC. IO /quote v1.5.2 approach. This capability is especially important for incremental updates in projects of large code.

Upgrade a dependency to the new major version

Now, let’s finish converting from using rsc. IO /quote to just using rsc. IO /quote/v3. We can expect some APIS to have been removed, renamed, or made incompatible changes due to changes in the main release. Reading the documentation, we can see that Hello is now HelloV3:

$ go doc rsc.io/quote/v3 package quote // import "rsc.io/quote" Package quote collects pithy sayings. func Concurrency()  string func GlassV3() string func GoV3() string func HelloV3() string func OptV3() string $Copy the code

(There is a known BUG in the output, which shows content that references the path incorrectly with /v3 discarded.)

We can update quote.hello () in our hello.go file to quotev3.hello () :

package hello

import quoteV3 "rsc.io/quote/v3"

func Hello() string {
	return quoteV3.Hello()
}

func Proverb() string {
	return quoteV3.Concurrency()
}
Copy the code

At this point, we no longer need to rename references, we can remove these:

package hello

import "rsc.io/quote/v3"

func Hello() string {
	return quote.Hello()
}

func Proverb() string {
	return quote.Concurrency()
}
Copy the code

Now let’s rerun the test to make sure everything works.

$go test PASS OK example.com/hello 0.014sCopy the code

Remove useless dependencies

IO /quote we have removed all use of rsc. IO /quote, but we can still see it in our go.mod file when we run the go list -m all command.

$go list -m all example.com/hello golang.org/x/text v0.3.0 rsc. IO /quote v1.5.2 rsc. IO /quote/v3 v3.1.0 rsc. IO /sampler V1.3.1 $cat go.mod module example.com/hello go 1.12 require (golang.org/x/text v0.3.0 // indirect rsc. IO /quote v1.5.2 Rsc. IO/v3.0.0 rsc. IO /sampler v1.3.1 // indirect) $Copy the code

Why is that? Because when we build a single package, such as go Build or Go Test, it’s easy to know which packages are missing or need to be added, but it’s hard to know which packages can be safely removed. Remove a dependency only after you have examined all packages and possible build tag combinations for a module. Normal build commands do not trap this information, and therefore cannot safely remove dependencies.

The go mod Tidy command helps clean up unwanted dependencies:

$go mod Tidy $go list -m all example.com/hello golang.org/x/text v0.3.0 rsc. IO /quote/v3 v3.1.0 rsc. IO /sampler v1.3.1 $ Cat go.mod module example.com/hello go 1.12 require (golang.org/x/text v0.3.0 // indirect rsc. IO /quote/v3 v3.1.0 Rsc. IO /sampler v1.3.1 // indirect) $go test PASS OK example.com/hello 0.020s $Copy the code

conclusion

The Go Modules feature is the future of dependency management in Go. Module functionality is now available in the current technology-supported versions of Go (currently Go1.11 and Go1.12).

This article introduces some workflows using Go Modules:

  • go mod initCreate a new module and initializego.modFile.
  • go buildandgo testCommand and some other package build commands to add necessary new dependencies togo.modFile.
  • go list -m allPrints the current module dependency.
  • go getCommand to change the version of a dependency (or to add a dependency).
  • go mod tidyRemove useless dependencies.

We encourage you to enable modules in your local development from now on, and add the go.mod and go.sum files to your project. You can send us BUG feedback or experience reports to provide feedback and help with the future evolution of Go dependency management.

Thank you for your feedback to help improve modules.