Introduce a.

Every dependency management solution must address the problem of choosing versions of dependencies. Many version selection algorithms that exist today attempt to identify the latest Greatest version of any dependency. This makes sense if you think that Semantic versioning will be applied correctly and that the social contract will be adhered to. In such cases, the “latest Max” version of the dependency should be the most stable and secure version, and should have backward compatibility with older versions. At least in the same Major Verion dependency tree.

Go decided to Go the other way, and Russ Cox spent a lot of time writing articles and speaking about the Go team’s Version Selection methods, known as Minimal Version Selection or MVS(Minimal Version Selection). In essence, the Go team believes that MVS provides the best solution for persistent and repeatable builds of the Go program. I encourage you to read this article to see why the Go team believes this.

In this article, I’ll do my best to explain THE MVS semantics, show an example of the Go language in action, and actually use the MVS algorithm.

2. MVS semantics

Naming Go’s dependency version selection algorithm “Minimal Version selection” is a bit of a misnomer, but once you see how it works, you’ll see that the name really fits. As I mentioned earlier, many selection algorithms select the “latest Max” version of a dependency. I like to think of MVS as an algorithm that selects the latest non-greatest version. This is not to say that MVS cannot choose “latest Max”, but as long as no dependency in the project requires “latest Max”, then this version is not required.

To better understand this, let’s create A situation where several modules (A, B, and C) depend on the same module (D), but each requires A different version.

The figure above shows how Modules A, B, and C independently need different versions of Module D and D respectively.

If I start A project that requires Module A, I also need Module D in order to build the code. Module D may have many versions to choose from. For example, suppose module D represents Sirupsen’s Logrus Module. I can ask Go to provide me with a list of all the existing (tagged) versions of Module D.

Listing 1:

$ go list -m -versions github.com/sirupsen/logrusGithub.com/sirupsen/logrus v0.1.0 v0.1.1 v0.2.0V0.3.0 v0.4.0 v0.4.1 v0.5.0 v0.5.1 v0.6.0 v0.6.1V0.6.2 v0.6.3 v0.6.4 v0.6.5 v0.6.6 v0.7.0 v0.7.1V0.7.2 v0.7.3 v0.8.0 v0.8.1 v0.8.2 v0.8.3 v0.8.4V0.8.5 v0.8.6 v0.8.7 v0.9.0 v0.10.0 v0.11.0 v0.11.1V0.11.2 v0.11.3 v0.11.4 v0.11.5 v1.0.0 v1.0.1 v1.0.3V1.0.4 v1.0.5 v1.0.6 v1.1.0 v1.1.1 v1.2.0 v1.3.0V1.4.0 v1.4.1 v1.4.2Copy the code

Listing 2 shows all versions of Module D that exist, and we see that the “latest and largest” version is shown as 1.4.2.

Which version of Module D should be selected for this project? There are really two options. The preferred option is to select the “latest” version (in the line with major version 1), which is V1.4.2. The second option is to select the version required for Module A, V1.0.6.

Dependency tools like DEP will be selected for v1.4.2 and will work with semantic versioning and social contract compliance. However, for some of the reasons explained here by Russ Cox, Go will honor Module A’s request and choose version 1.0.6. In the currently required version set of all dependencies for projects that require modules, Go selects the “minimum” version. In other words, only Module A now needs Module D, and Module A has specified that the version it requires is v1.0.6, and there is only v1.0.6 in the required version set, so the version of Module D selected by Go is it.

What if I introduce new code that requires the project to import Module B? After importing Module B into the project, Go will upgrade the project’s Module D version from V1.0.6 to v1.2.0. Go again selects the “minimum” version of Module D in the current required version set of project dependencies Module A and B (v1.0.6 and v1.2.0).

What if I reintroduce new code that requires the project to import Module C? Go will select the latest version (v1.3.2) from the current set of required versions (V1.0.6, V1.2.0, v1.3.2). Note that version V1.3.2 is still the “smallest” version of Module D (v1.4.2), not the “latest largest” version.

Finally, what happens if you delete the code you just added that relies on Module C? Go locks the project to module D version V1.3.2. Demoting to v1.2.0 would be an even bigger change, and Go knows that v1.3.2 is healthy and stable, so v1.3.2 is still the “latest non-greatest” version of Module D. In addition, the module file (go.mod) only maintains snapshots, not logs. There is no information about historical undo or downgrade.

This is why I like to think of MVS as the algorithm that selects the latest non-greatest version of the Module. Hopefully you can now understand why Russ Cox chose the name “minimal” when naming the algorithm.

Sample project

With that in mind, I’ll use a sample project to show you how the Go and MVS algorithms actually work. In this project, Module D will be represented by Logrus Module, and the project will rely directly on RethinkDB-Go (moduleA) and Golib (moduleB) Modules. Rethinkdb-go and Golib Modules rely directly on Logrus Modules, and each module requires a different version of Logrus, none of which is the “latest” version of Logrus.

The figure above shows the independent relationships between the three Modules. First, I’ll create the project, initialize the Module, and then load VS Code.

Listing 2:

$ cd $HOME$ mkdir app$ mkdir app/cmd$ mkdir app/cmd/db$ touch app/cmd/db/main.go$ cd app$ go mod init app$ code .Copy the code

Listing 2 shows all the commands to run. After running these commands, the following Code should appear in VS Code.

The figure above shows the project structure and what the Module file should contain. With this in mind, it’s time to add code that uses the Rethinkdb-go Module.

Listing 3:https://play.golang.org/p/bc5I0Afxhvc

01 package main0203 import (04     "context"05     "log"0607     db "gopkg.in/rethinkdb/rethinkdb-go.v5"08)0910 func main() {11     c, err := db.NewCluster([]db.Host{{Name: "localhost", Port: 3000}}, nil)12 if err ! = nil {13         log.Fatalln(err)14}1516 if _, err = c.Query(context.Background(), db.Query{}); err ! = nil {17         log.Fatalln(err)18}19}Copy the code

Listing 3 introduces the Major version V5 of the RethinkDB-Go Module. After you add and save this code, Go finds, downloads, and extracts the Module, and updates the go.mod and go.sum files.

Listing 4:

01 module app0203 go 1.130405 the require gopkg. In/rethinkdb/rethinkdb - go. V5 version 5.0.1Copy the code

Listing 4 shows that Go.mod requires rethinkdb-go Module as a direct dependency and selects v5.0.1, which is the “latest and largest version” of the module.

Listing 5:

.Github.com/sirupsen/logrus v1.0.6 h1: hcP1GmhGigz O7h1WVUM5KklBp1JoNS9FggWKdj/j3s =Github.com/sirupsen/logrus v1.0.6 / go mod h1: pMByvHTf9Beacp5x1UXfOR9xyW/antxmhjmpg0dezc = 9.Copy the code

Listing 5 shows the two lines in the go.sum file that introduce logrus Module v1.0.6. At this point, you can see that the MVS algorithm has selected the “minimum” version of logrus Module required to satisfy the rethinkDB-Go Module specification. Remember that the “latest and largest” version of Logrus Module is 1.4.2.

Note: The go.sum file should not be used to understand dependencies. The versioning I did above is wrong, and I’ll show you the correct way to determine the version your project is using later.

The figure above shows which version of Logrus Module Go will use to build the project.

Next, I’ll add code that introduces a dependency on the Golib Module.

Listing 6:https://play.golang.org/p/h23opcp5qd0

01 package main0203 import (04     "context"05     "log"0607     "github.com/Bhinneka/golib"08     db "gopkg.in/rethinkdb/rethinkdb-go.v5"09)1011 func main() {12     c, err := db.NewCluster([]db.Host{{Name: "localhost", Port: 3000}}, nil)13 if err ! = nil {14         log.Fatalln(err)15}1617 if _, err = c.Query(context.Background(), db.Query{}); err ! = nil {18         log.Fatalln(err)19}2021     golib.CreateDBConnection("")22}Copy the code

Listing 6 adds lines 07 and 21 to the program. After Go finds, downloads, and unpacks the Golib Module, the following changes are displayed in the go.mod file.

Listing 7:

01 module app0203 go 1.130405 require (06 github.com/Bhinneka/golib v0.0.0 dc569916cba - 20191209103129-107 gopkg. In/rethinkdb/rethinkdb - go. V5 version 5.0.108)Copy the code

Listing 7 shows that the go.mod file has been modified to include a “last Max” version dependency for Golib Module, which happens to have no semantic version tag.

Listing 8:

.Github.com/sirupsen/logrus v1.0.6 h1: hcP1GmhGigz O7h1WVUM5KklBp1JoNS9FggWKdj/j3s =Github.com/sirupsen/logrus v1.0.6 / go mod h1: pMByvHTf9Beacp5x1UXfOR9xyW/antxmhjmpg0dezc = 9Github.com/sirupsen/logrus v1.2.0 h1: juTguoYk5qI21pwyTXY3B3Y5cOTH3ZUyZCg1v/mihuo =Github.com/sirupsen/logrus v1.2.0. / go mod h1: LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo =.Copy the code

Listing 8 shows four lines in the go.sum file, which now includes versions V1.0.6 and V1.2.0 of the Logrus Module. Looking at the two versions listed in the go.sum file brings up two problems:

• Why are two versions listed in the go.sum file? • Which version will be used when Go performs builds?

Bryan Mills of the Go team has a good answer for the reasons for listing both versions in the Go.sum file.

“The go.sum file still contains the older version (1.0.6) because its requirement to pass dependencies may affect selected versions of other modules. We really only needed to provide a checksum for the go.mod file because go.mod declared these delivery requirements, but because Go Mod Tidy wasn’t precise enough, we ended up keeping the checksum of the source code as well.” golang.org/issue/33008

There is still the question of which version of Logrus Module will be used when building the project. To determine exactly which modules and versions will be used, do not look at the go.sum file, but use the go list command.

Listing 9:

$ go list -m all | grep logrusGithub.com/sirupsen/logrus v1.2.0Copy the code

Listing 9 shows the V1.2.0 version of the Logrus Module that will be used when building the project. The -m flag instructs the Go List to list modules instead of packages.

Look at the Module diagram to learn more about the project’s requirements for the Logrus Module.

Listing 10:

$ go mod graph | grep logrusgithub.com/sirupsen/[email protected] github.com/pmezard/[email protected]github.com/sirupsen/[email protected] github.com/stretchr/[email protected]github.com/sirupsen/[email protected] github.com/stretchr/[email protected]github.com/sirupsen/[email protected] golang.org/x/[email protected]github.com/sirupsen/[email protected] golang.org/x/[email protected]Gopkg. In/rethinkdb/[email protected] github.com/sirupsen/[email protected]github.com/sirupsen/[email protected] github.com/konsorten/[email protected]github.com/sirupsen/[email protected] github.com/davecgh/[email protected]github.com/Bhinneka/[email protected] github.com/sirupsen/[email protected]github.com/prometheus/[email protected] github.com/sirupsen/[email protected]Copy the code

Listing 10 shows the logrus Module relationship in the project. I’ll just extract the lines that show the dependency requirements on Logrus.

Listing 11:

Gopkg. In/rethinkdb/[email protected] github.com/sirupsen/[email protected]github.com/Bhinneka/[email protected] github.com/sirupsen/[email protected]github.com/prometheus/[email protected] github.com/sirupsen/[email protected]Copy the code

In Listing 11, these lines show that all three modules (RethinkDB-Go, Golib, and Common) require logrus Modules. Thanks to the go list command, I know that the minimum version required is V1.2.0.

The figure above shows which version of Logrus Module Go will now use to build the code in the project.

Go Mod Tidy

Run Go Mod Tidy to make sure the Module file is up to date and accurate before committing/pushing the code back to the repository. The code you build, run, or test locally will affect how Go updates the contents of the Module file at any time. Running Go Mod Tidy will ensure that the project has an accurate and complete snapshot of what it needs, which will help others on your team and your CI/CD environment.

Listing 12:

$ go mod tidygo: finding github.com/Bhinneka/golib latestgo: finding github.com/bitly/go-hostpool latestgo: finding github.com/bmizerany/assert latestCopy the code

Listing 12 shows the output after running Go Mod Tidy. You should see two new dependencies in the output. This will change the Module file.

Listing 13:

01 module app0203 go 1.130405 require (06 github.com/Bhinneka/golib v0.0.0 dc569916cba - 20191209103129-107 github.com/bitly/go-hostpool v0.0.0-20171023180738-A3a6125DE932 / indirect08 github.com/bmizerany/assert v0.0.0-20160611221934 - b7ed37b82869 / / indirect09 gopkg. In/rethinkdb/rethinkdb - go. V5 version 5.0.110)Copy the code

Listing 13 shows the Go-HostPool and Assert Modules listed as indirect modules needed to build the project. They are listed here because these projects are currently incompatible with the Module mechanism. In other words, the go.mod file does not exist in any tag version or “latest” version of the master for these projects.

Why are these Modules included after running Go Mod Tidy? I can find out using the go mod Why command.

Listing 14:

$ go mod why github.com/hailocab/go-hostpool# github.com/hailocab/go-hostpoolapp/cmd/dbgopkg.in/rethinkdb/rethinkdb-go.v5github.com/hailocab/go-hostpool------------------------------------------------$ go mod why github.com/bmizerany/assert# github.com/bmizerany/assertapp/cmd/dbgopkg.in/rethinkdb/rethinkdb-go.v5github.com/hailocab/go-hostpoolgithub.com/hailocab/go-hostpool.testgithub.com/bmizerany/assertCopy the code

Listing 14 shows why the project indirectly needs these Modules. The rethinkdb-Go Module requires the Go-HostPool Module, and the Go-HostPool Module requires an Assert Module.

Upgrade dependencies

The project has three dependencies, each of which requires logrus Module, of which the v1.2.0 version is currently being selected. At some point in the project lifecycle, it becomes important to upgrade direct and indirect dependencies to ensure that the code required by the project is up to date and available to take advantage of new features, bug fixes, and security patches. To upgrade, Go provides the Go Get command.

Before running the dependencies for the Go Get upgrade project, you need to consider several options.

Use MVS to upgrade only necessary direct and indirect dependencies

I recommend starting with this upgrade until you learn more about the project and module. This is the most conservative form of go get.

Listing 15:

$ go get -t -d -v ./...Copy the code

Listing 15 shows how to use the MVS algorithm to upgrade those required dependencies. Here are some definitions of command line type selection in commands.

•-t flag: Consider modules needed to build tests. •-d flag: Download the source code for each Module, but do not build or install them. •-v flag: displays detailed output. •. /… : Perform these operations throughout the source tree and update only the required dependencies.

Running this command on the current project will not result in any changes because the project is already up to date and has the minimum version required to build and test it. That’s because I just ran Go Mod Tidy, the project is new.

Upgrade only necessary direct and indirect dependencies with the latest Max release

This upgrade increases the dependency of the entire project from “minimum” to “latest maximum.” All you need to do is add the -u flag to the command line.

Listing 16:

$ go get -u -t -d -v ./...go: finding golang.org/x/net latestgo: finding golang.org/x/sys latestgo: finding github.com/hailocab/go-hostpool latestgo: finding golang.org/x/crypto latestgo: finding github.com/google/jsonapi latestgo: finding gopkg.in/bsm/ratelimit.v1 latestgo: finding github.com/Bhinneka/golib latestCopy the code

Listing 16 shows the output of running the go get command with the -u flag. This output does not tell the truth. What happens if I ask the go list command which version of Logrus Module is now used to build the project?

Listing 17:

$ go list -m all | grep logrusGithub.com/sirupsen/logrus v1.4.2Copy the code

Listing 17 shows how to select the “latest” Logrus. To make this choice more explicit, changes have been made to the go.mod file.

Listing 18:

01 module app0203 go 1.130405 require (06 github.com/Bhinneka/golib v0.0.0 dc569916cba - 20191209103129-107 github.com/bitly/go-hostpool v0.0.0-20171023180738-A3a6125DE932 / indirect08 github.com/bmizerany/assert v0.0.0-20160611221934 - b7ed37b82869 / / indirect09 github.com/cenkalti/backoff v2.2.1 + incompatible / / indirect10 github.com/golang/protobuf v1.3.2 / / indirect11     github.com/jinzhu/gorm v1.9.11 // indirect12     github.com/konsorten/go-windows-terminal-sequences v1.0.2 // indirect13 github.com/sirupsen/logrus v1.4.2 / / indirect15 golang.org/x/crypto v0.0.0-20191206172530-e9b2fee46413 / indirect15     golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553 // indirect16     golang.org/x/sys v0.0.0-20191210023423-ac6580df4449 // indirect17 gopkg. In/rethinkdb/rethinkdb - go. V5 version 5.0.118)Copy the code

Listing 18 shows in line 13 that version V1.4.2 is now the selected version of the Logrus Module in the project. When building a project, Go pays attention to this line in the Module file. Even with the logrus Module dependency change code removed, version V1.4.2 of the project is now locked. Keep in mind that the downgrade will be a bigger change and v1.4.2 will not be affected.

What changes can be seen in the go.sum file?

Listing 19:

Github.com/sirupsen/logrus v1.0.6 / go mod h1: pMByvHTf9Beacp5x1UXfOR9xyW/antxmhjmpg0dezc = 9Github.com/sirupsen/logrus v1.2.0 h1: juTguoYk5qI21pwyTXY3B3Y5cOTH3ZUyZCg1v/mihuo =Github.com/sirupsen/logrus v1.2.0. / go mod h1: LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo =Github.com/sirupsen/logrus v1.4.2 h1: SPIRibHv4MatM3XXNO2BJeFLZwZ2LvZgfQ5 + UNI2im4 =Github.com/sirupsen/logrus v1.4.2. / go mod h1: tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE =Copy the code

Listing 19 shows all three versions of Logrus represented in the go.sum file. As Explained by Bryan above, this is because delivery requirements can affect selected versions of other Modules.

The figure above shows which version of Logrus Module Go will now use to build the code in the project.

Upgrade all direct and indirect dependencies with the latest and largest version

You can change your./… The option is replaced with all to upgrade all direct and indirect dependencies, including those that are not needed when building the project.

Listing 20:

$ go get -u -t -d -v allGo: downloading github.com/mattn/go-sqlite3 v1.11.0Go: extracting github.com/mattn/go-sqlite3 v1.11.0go: finding github.com/bitly/go-hostpool latestgo: finding github.com/denisenkom/go-mssqldb latestgo: finding github.com/hailocab/go-hostpool latestgo: finding gopkg.in/bsm/ratelimit.v1 latestgo: finding github.com/google/jsonapi latestgo: finding golang.org/x/net latestgo: finding github.com/Bhinneka/golib latestgo: finding golang.org/x/crypto latestgo: finding gopkg.in/tomb.v1 latestgo: finding github.com/bmizerany/assert latestgo: finding github.com/erikstmartin/go-testdb latestgo: finding gopkg.in/check.v1 latestgo: finding golang.org/x/sys latestgo: finding github.com/golang-sql/civil latestCopy the code

Listing 20 shows how many dependencies are now found, downloaded, and extracted for the project.

Listing 21:

Added to Module FileCloud.google.com/go v0.49.0 / / indirectD7a30a10f73 github.com/denisenkom/go-mssqldb v0.0.0-20191128021309-1 / / indirectGithub.com/google/go-cmp v0.3.1 / / indirectGithub.com/jinzhu/now v1.1.1 / / indirectGithub.com/lib/pq v1.2.0 / / indirectGithub.com/mattn/go-sqlite3 v2.0.1 + incompatible / / indirectGithub.com/onsi/ginkgo v1.10.3 / / indirectGithub.com/onsi/gomega v1.7.1 / / indirectGithub.com/stretchr/objx v0.2.0 / / indirectGoogle.golang.org/appengine v1.6.5 / / indirectGopkg. / check in. V1 v1.0.0-20190902080502-41 f04d3bba15 / / indirectGopkg. In/yaml. V2 v2.2.7 / / indirectRemoved from Module FileGithub.com/golang/protobuf v1.3.2 / / indirectCopy the code

Listing 21 shows the changes to the go.mod file. More modules were added and one module was removed.

Note: If you use vendor, the go mod vendor command will strip the test file from the vendor folder.

In general, do not use the all or -u option when upgrading a project’s dependencies with Go Get. Do upgrade only the modules you need, and use the MVS algorithm to select those modules and their versions. Manually change to a specific Module version if necessary. Manual changes can be made by manually editing the go.mod file, which I’ll show you in a future article.

Reset dependencies

If you are not happy with the module and version you selected at any time, you can always reset the selection by deleting the Module file and running Go Mod Tidy again. This is especially an option when the project is young and unstable. Once the project is stable and released, I hesitate to reset the dependencies. As I mentioned above, module versions may be set over time, and you need long lasting, repeatable builds.

Listing 22:

$ rm go.*$ go mod init <module name>$ go mod tidyCopy the code

Listing 22 shows the command that allows MVS to execute all the selections again from scratch. I’ve been doing this throughout the writing of this article to reset the project and provide the code listing for this article.

Conclusion six.

In this article, I explain THE MVS semantics and show a real-world example of the Go and MVS algorithms in action. I also showed you some Go commands that can provide you with information when you encounter an unknown problem. As you add more and more dependencies to your project, you may encounter some extreme situations. This is because the Go ecosystem is 10 years old, and all existing projects need more time to meet Module requirements.

In future articles, I’ll discuss using different major versions of dependencies in the same project and how to manually retrieve and lock specific versions of dependencies. Now, I hope you have more trust in the Module and Go tools, and a clearer understanding of how MVS selects versions over time. If you run into any problems, there’s a group of people on the # Module group’s Gopher Slack who are willing to help.

This article is translated from Modules Part 03: Minimal Version Selection “https://www.ardanlabs.com/blog/2019/12/modules-03-minimal-version-selection.html.

Recommended reading

  • How to visualize Go Module dependencies

  • Why use Go Module Proxy?

If you like this article, please pay attention to “Go Language Chinese” :

Go Language Chinese opens wechat learning exchange group, welcome to add wechat: 274768166