Brief introduction:The problem stems from the fact that we want to make a change to the dubbo-go module path, replacing the previous “red” with dubbo.apache.org/dubbo-go/v3.

The author | Dong Jianhui, ShengAoFei source | alibaba cloud native public number

The problem background

The problem stems from the fact that we want to make a change to the dubbo-go module path, replacing the previous “red” with dubbo.apache.org/dubbo-go/v3. First, we did the pathmapping and placed a dubbogo/v3 file under dubbo.apache.org with the following contents:

<html> <head> <meta name="go-import" content="dubbo.apache.org/dubbo-go/v3 git <https://github.com/apache/dubbo-go>"> < meta name = "go - the source" content = "dubbo.apache.org/dubbo-go/v3 git {/ dir} < https://github.com/apache/dubbo-go/tree/3.0 > {< https://github.com/apache/dubbo-go/blob/3.0 / dir} / {file} {line} # L > "> < meta HTTP - equiv =" refresh "content =" 0; url=https://pkg.go.dev/dubbo.apache.org/dubbo-go/v3"> </head> <body> <p>Redirecting to <a href="<https://pkg.go.dev/dubbo.apache.org/dubbo-go/v3>">pkg.go.dev/dubbo.apache.org/dubbo-go/v3</a>... </p> </body> </html>

Next, we changed the module of go.mod and all its corresponding imports, as well as the module path used by all submodules to reference dubbo-go.

Problem analysis

​

After the above modification, we found CI failed when we mentioned PR. After further log checking, we determined that CI made an error when running the integration test. The specific error message is as follows:

​

The execution logic of this section is that we want to use Docker to mirror the integration test content in Dubbo-Go and start the container for testing. The images used in the packaging Dockerfile path in github.com/apache/dubbo-go/test/integrate/dubbo/go-server directory, controlled STEP identification error log, We can locate the specific error in the following two steps:

#... # STEP 9 RUN test ${PR_ORIGIN_REPO} && go mod edit -replace=dubbo.apache.org/dubbo-go/v3=github.com/{PR_ORIGIN_REPO}@${PR_ORIGIN_COMMITID} || go get -u dubbo.apache.org/dubbo-go/v3@develop # ... # STEP 11 RUN go mod tidy && go install github.com/apache/dubbo-go/test/integrate/dubbo/go-server

​

In STEP 9, we usego mod edit -replaceReplace the dependency path for Dubbogo with the repository address and Commit ID from which the PR request was made. On this basis, when the mirror build goes to Step 11, try using itgo mod tidyAn error occurred while pulling the bag.

​

Looking back at the error log, we can see that:

​

Since we have only specified a COMMIT ID, during the actual run of the Go Mod, it will generate a hypothetical version number for us (see the problem extension at the end of this article for more information on the hypothetical version number), which will capture the last valid tag of the remote repository as v1.4.1 [Note: The remote repository here is github.com/Mulavar/dubbo-go, and this is my own dubbo-go branch. This branch was forked earlier, and its last tag is v1.4.1, which is automatically increased to v1.4.2. The main version is V1. But in previous, our dubbogo module path is declared as dubbo.apache.org/dubbogo/v3, major version is v3, this led to go mod edit – before and after the replace rely on major version is v1 and v3 respectively, An inconsistency occurred and an error occurred when actually pulling the Dubbogo dependency.

Problem solving

In the section of problem analysis, we locate this problem in STEP 9 of mirror construction, that is:

# STEP 9
RUN test ${PR_ORIGIN_REPO} && go mod edit -replace=dubbo.apache.org/dubbo-go/v3=github.com/{PR_ORIGIN_REPO}@${PR_ORIGIN_COMMITID} || go get -u dubbo.apache.org/dubbo-go/v3@develop

At this step, when we use Go Mod Edit-Replace, we replace a V3 version of the Go module with a V1 version of the module, resulting in inconsistencies in the main version and errors. Therefore, we only need to specify that the main version of the replaced module is also v3 when replacing the dependency path. We also add the v3 suffix to the dependency path of the module after the go mod edit-replace, as shown below. Go mod edit-replace =dubbo.apache.org/dubbo-go/v3=github.com/{PR_ORIGIN_REPO}@${PR_ORIGIN_COMMITID} flicker Go mod edit-replace =dubbo.apache.org/dubbo-go/v3=github.com/${PR_ORIGIN_REPO}/v3@${PR_ORIGIN_COMMITID} After fixing the problem we submit the code to view CI, in the log information printed by Step 8, it can be seen that the replaced Dubbogo path has more v3, and the version number (v3.0.0-20210509140455-2574EAB5AD0B) followed when pulling the package is also v3, which successfully pulled the dependency.

​

Problems develop

​

1. Semantic Import Versioning

​

When we use Go Modules to build the dependencies of the project, we need to declare the version number we are using for the dependencies of the Go project. Considering that compatibility is always the most important and painful issue for developers every time we release a new version, so Go is approvedSemantic Import VersioningTo establish a standard version number to ensure that each developer can specify the dependency version to be used according to their project requirements. According to Go’s semantic import version control guidelines, the version number consists of three parts:

​

Note: The above picture and parts are from the article “Go Modules in detail”.

A Major version is a new version, even if the new version is incompatible with the old version. Minor version refers to an iteration within a larger version, which is used to increment features when they are added. Patch version is the most granular version update, which represents some bug fixes. There are two ways to specify a major version, either by declaring it directly as above, or by adding a version suffix at the end of the project path. For example, the Dubbogo 3.0 version declares the module dubbo.apache.org/dubbo-go/v3, where v3 is a major version declaration.

2. pseudo-version

​

Go encourages us to specify explicit version numbers when using dependencies, such as the pair declared in Dubbogo’s go.modcloud.google.com/goThe dependence of:

​

But considering that there are scenarios where we want to use an unpublished version of a module dependency that only has a known COMMIT ID, such as the github.com/Microsoft/go-winio dependency declared by Dubbogo, You can use a pseudo version instead of the real version. Assume the format of the version number is:

// (2) VX.Y.(Z+1)-0. YYYYYMMDDHHMMSS-ABCDEF123456 // (3) vX.Y.(Z+1)-0.yyyymmddhhmmss-abcdef123456+incompatible // (4) vX.Y.Z-pre.0.yyyymmddhhmmss-abcdef123456 // (5) vX.Y.Z-pre.0.yyyymmddhhmmss-abcdef123456+incompatible

It can be seen that pseudo-version is divided into three parts by short slashes. The first part, X.Y.Z, is consistent with what was mentioned in Section 4.1, namely, major version, minor version and patch version, respectively. The second part is a timestamp in the format yyyymmddhhhmmss, and the third part is a 12-bit commit hash that can be specified manually or automatically generated by go if it is not. Regarding pseudo-version, the generation rules for Go are as follows:

  • Query the latest tag for the project corresponding to the major version (or version v1 if the project dependency path does not have an explicit v2/v3 suffix).
  • If there is a tag, increment patch version with the latest tag.
  • No tag, automatically generated according to the project path with the suffix version number (such as V3 automatically generated a 3.0.0 startpseudo-version).

Following this rule, looking back at the problem analysis and problem solving above, we can see why the pseudo-version of dubbo-go is v1.4.2 at the beginning. This is exactly because Go thinks that the main version of dubbogo after replace is V1. We searched for the latest tag under this major version and incrementing patch version, which resulted in the inconsistency between the previous and the previous major versions. When we added v3 to the version path, GO could not find the tag under the corresponding major version, so v3.0.0 was automatically generated for us and CI was passed.

3. Go modules are nested

Unlike Java, Go does not actually have the concept of submodules. Sometimes, though, we’ll see a repo with multiple Go modules, such as a project with a Go.mod in its root directory and some subdirectories with Go.mod in them. This is called a nested module in Go modules, rather than a parent-child module, which means that the two modules have no relationship and are independent of each other. So when do you need a single repo with multiple modules? Generally speaking, there are two situations when we would consider using a single-repo multi-module development format.

  1. A nested module changes very frequently and needs to be published frequently.
  2. The nested modules depend only on a fixed version of the root module.

In both cases, the essence is that there is no strong version binding relationship between the two modules. However, due to some other reasons, they need to be placed under a single RPEO, thus forming a situation of multiple modules with a single REPO.

4. Dubbogo static mapping file content resolution

Dubbo-go uses the mapping of static files to achieve module redirection. The contents of static files are as follows: The core of the files are the META tags GO-IMPORT and GO-Source.

<html> <head> <meta name="go-import" content="dubbo.apache.org/dubbo-go/v3 git <https://github.com/apache/dubbo-go>"> < meta name = "go - the source" content = "dubbo.apache.org/dubbo-go/v3 git {/ dir} < https://github.com/apache/dubbo-go/tree/3.0 > {< https://github.com/apache/dubbo-go/blob/3.0 / dir} / {file} {line} # L > "> < meta HTTP - equiv =" refresh "content =" 0; url=https://pkg.go.dev/dubbo.apache.org/dubbo-go/v3"> </head> <body> <p>Redirecting to <a href="<https://pkg.go.dev/dubbo.apache.org/dubbo-go/v3>">pkg.go.dev/dubbo.apache.org/dubbo-go/v3</a>... </p> </body> </html>

​

1) go to import

The go-import is used to tell the go get where to find the source code. The content is divided into three parts:

  • dubbo.apache.org/dubbo-go/v3Module declaration for the project.
  • git: The version control tool used.
  • <https://github.com/apache/dubbo-go>: Tells Go Get where to find the source code for the project.

2) the go – source

Go-Source generates a specific Go Doc (now pkg.go.dev) document for the project. There are four parts. The first two parts are the module declaration and version control tool for the project, and the last two parts serve the following functions respectively:

  • {/ dir} > < https://github.com/apache/dubbo-go/tree/3.0: Declares the location of the source code for the project.
  • {< https://github.com/apache/dubbo-go/blob/3.0 / dir} / {file} {line} > # L: Mapping documents and code to help you jump to the corresponding content when you click on the document’s directory tree.

In https://pkg.go.dev/dubbo.apache.org/dubbo-go/v3, for example, we click one of the file:

It will jump to the corresponding document for the corresponding file.

​

If you are interested in the Apache/Dubbo-Go project, please search the Nailing group number 31363295 and join the Nailing group!

The resources

  • “The Go Blog – Using The Go Modules: https://blog.golang.org/using-go-modules
  • “Go Modules Reference: https://golang.org/ref/mod
  • “How Go Module launch v2 and above version” : https://blog.cyeam.com/go/2019/03/12/go-version
  • “Go Modules, rounding: https://www.sulinehk.com/post/go-modules-details/

Author’s brief introduction

Jianhui Dong (github @mulavar) is a recent graduate of the VLIS lab at Zhejiang University. He is currently a development engineer for the ape tutoring service. Sheng Aofei (github @aofei), author of the goproxy.cn project, has submitted a lot of PR to GO source code (such as mod tools).

Copyright Notice:The content of this article is contributed by Aliyun real-name registered users, and the copyright belongs to the original author. Aliyun developer community does not own the copyright and does not bear the corresponding legal liability. For specific rules, please refer to User Service Agreement of Alibaba Cloud Developer Community and Guidance on Intellectual Property Protection of Alibaba Cloud Developer Community. If you find any suspected plagiarism in the community, fill in the infringement complaint form to report, once verified, the community will immediately delete the suspected infringing content.