This article first in my blog, if you feel useful, welcome to like collection, let more friends see. Related video (first recorded)

Recently, I developed a very simple gadget with less than 200 lines of code. Today, I’ll talk a little bit about it. What kind of tool is this? It is a tool for visualizing Go Module dependencies.

Why develop

Why did you want to develop this tool? There are two main reasons:

First, I often see people discussing the Go Module in the community recently. So, I spent some time doing some research. During that time, I encountered a need to clearly identify the relationships between dependencies in a module. After some digging, I found the Go Mod Graph.

The effect is as follows:

$go mod graph github.com/poloxue/testmod golang.org/x/[email protected] github.com/poloxue/testmod RSC. IO/quote/[email protected] Github.com/poloxue/testmod RSC. IO/[email protected] golang.org/x/[email protected] golang.org/x/[email protected] RSC. IO/quote/[email protected] RSC. IO/[email protected] RSC. IO/[email protected] golang.org/x/[email protected] RSC. IO/[email protected] golang.org/x/[email protected]Copy the code

The format of each line is module-dependent, which basically meets the requirements, but it’s still not that intuitive.

Second, I used to use DEP for package management in a project I was working on. So I read about it and read the official document carefully. One of the sections describes how to visualize dependencies.

The package diagram given in the document:

When I see this picture, my eyes light up instantly. The graphics are excellent and the relationship between different dependencies is clear at a glance. Isn’t that what I wanted? 666, give it a thumbs up.

But… The problem is, the Go Mod doesn’t have that capability. How to do?

How to implement

Let’s see if someone has already done it. I did an Internet search, but I couldn’t find it. Is that self-fulfilling? Should be able to draw lessons from the idea of DEP?

Here is how deP dependencies are visualized:

# linux
$ sudo apt-get install graphviz
$ dep status -dot | dot -T png | display

# macOS
$ brew install graphviz
$ dep status -dot | dot -T png | open -f -a /Applications/Preview.app

# Windows
> choco install graphviz.portable
> dep status -dot | dot -T png -o status.png; start status.png
Copy the code

Here’s how it works on all three systems, all of which have a software package, Graphviz, installed. According to the name, this should be a software used to achieve visualization, that is, to draw pictures. In fact, check out its website.

The use of look at it again and found that is, by means of combination of piping and the front part of the basic same, is dep status – dot | dot – T PNG. At the back of the part is different in different systems, Linux is the display, MacOS is open – f – a/Applications/Preview app, Window is start status. The PNG.

A little analysis will show you that the first is to generate the image and the second is to display the image. Since the image display commands vary from system to system, the rest is different.

Now the focus of the concern in front, namely the dep status | – dot dot -t PNG stem what, what exactly is it the drawing? As a rough guess, dot-T PNG is an image generated from data provided by DEP Status-dot. Take a look at the deP status-dot implementation.

$ dep status -dot
digraph {
	node [shape=box];
	2609291568 [label="github.com/poloxue/hellodep"];
	953278068 [label="RSC. IO/quote \ nv3.1.0"];
	3852693168 [label="RSC. IO/sampler \ nv1.0.0"]; 2609291568 - > 953278068; 953278068 - > 3852693168; }Copy the code

At first glance, the output looks like a piece of code that doesn’t know what it is. This should be the language graphviz uses to draw graphs. Is there still some learning to do? Of course not. It’s very easy to use here. Just apply it.

So let’s try to analyze this, because I don’t care about the first two lines, but this is kind of a graphviz specific way of saying what graph we’re going to draw. Our main concern is getting the data in the right form.

2609291568 [label="github.com/poloxue/hellodep"];
953278068 [label="RSC. IO/quote \ nv3.1.0"];
3852693168 [label="RSC. IO/sampler \ nv1.0.0"]; 2609291568 - > 953278068; 953278068 - > 3852693168;Copy the code

As you can see, there are two types of structures: associating ids with dependencies, and representing relationships between dependencies by ID and ->.

Following the above conjecture, we can try to draw a simple graph to show that module A depends on module B. Execute the following command to send the drawing code to the dot command through the each pipe.

$ echo 'digraph { node [shape=box]; 1 [label="a"]; 2 [label="b"]; 1 - > 2; } ' | dot -T png | open -f -a /Applications/Preview.app 
Copy the code

The effect is as follows:

Drawing a dependency diagram is surprisingly simple.

If you look at this, it’s pretty easy to spot the problem. We just need to transform the output of the Go Mod Graph into a similar structure to achieve the visualization.

Introduction to development Process

Next, develop this widget, which I’ll call modv, which stands for Module Visible. Project source code is located in Poloxue /modv.

Receive the input of the pipe

First check whether the data input pipeline is normal.

Our goal is to pipe data to modVs in a similar way to the way we draw in DEP. Therefore, check os.stdin first, that is, check whether the standard input status is normal and whether it is piped.

Here is the code for the main function, located in main.go.

func main(a) {
	info, err := os.Stdin.Stat()
	iferr ! =nil {
		fmt.Println("os.Stdin.Stat:", err)
		PrintUsage()
		os.Exit(1)}// Whether it is piped
	if info.Mode()&os.ModeNamedPipe == 0 {
		fmt.Println("command err: command is intended to work with pipes.")
		PrintUsage()
		os.Exit(1)}Copy the code

Once we confirm that all is well with the input device, we can proceed to the process of reading, parsing, and rendering the data.

	mg := NewModuleGraph(os.Stdin)
	mg.Parse()
	mg.Render(os.Stdout)
}
Copy the code

Next, start looking at how to implement the flow of data processing.

Abstract implementation structure

Define a structure and roughly define the entire process.

type ModGraph struct {
	Reader io.Reader  // Read the data stream
}

func NewModGraph(r io.Reader) *ModGraph {
    return &ModGraph{Reader: r}
}

// Perform data processing conversion
func (m *ModGraph) Parse(a) error {}

// Result rendering and output
func (m *ModGraph) Render(w io.Writer) error {}
Copy the code

Take a look at the go Mod graph’s output, which looks like this:

Github.com/poloxue/testmod golang.org/x/[email protected] github.com/poloxue/testmod RSC. IO/quote/[email protected]...Copy the code

Each row is structured as a module dependency. The goal now is to parse it into the following structure:

digraph { node [shape=box]; 1 github.com/poloxue/testmod; 2 golang.org/x/[email protected]; 3 RSC. IO/quote/[email protected]; 1 - > 2; 1 - > 3; }Copy the code

As mentioned earlier, there are two different constructs, namely, the relation between modules and ids, and the relation between modules that the module ID represents. Add two members to the ModGraph structure to represent them.

type ModGraph struct {
	r io.Reader  // Data stream read instance, in this case os.stdin
 
	// The mapping between the name and ID of each item
	Mods         map[string]int
	// ID and dependent ID relational mapping, an ID may depend on multiple items
	Dependencies map[int] []int
}
Copy the code

Note that after adding two map members, remember to initialize them in NewModGraph.

Mod Graph output parsing

How to parse it?

At this point, the objective is clear. The input data is parsed into the Mods and Dependencies members in the Parse method.

To facilitate data reading, we first create a new bufReader based on the bufio reader,

func (m *ModGraph) Parse(a) error {
	bufReader := bufio.NewReader(m.Reader)
	...
Copy the code

To make it easier to parse the data line by line, we loop through the data in OS.stdin line by line through bufReader’s ReadBytes() method. Then, each row of data is sliced by space to obtain two terms of the dependency relationship. The code is as follows:

for {
	relationBytes, err := bufReader.ReadBytes('\n')
	iferr ! =nil {
		if err == io.EOF {
			return nil
		}
		return err
	}

    relation := bytes.Split(relationBytes, []byte(""))
    // module and dependency
    mod, depMod := strings.TrimSpace(string(relation[0])), strings.TrimSpace(string(relation[1))... }Copy the code

The next step is to organize the resolved Dependencies into the Mods and Dependencies members. The module ID is the simplest way to implement the rule, increment from 1. The implementation code is as follows:

modId, ok := m.Mods[mod]
if! ok { modId = serialID m.Mods[mod] = modId serialID +=1
}

depModId, ok := m.Mods[depMod]
if! ok { depModId = serialID m.Mods[depMod] = depModId serialID +=1
}

if _, ok := m.Dependencies[modId]; ok {
	m.Dependencies[modId] = append(m.Dependencies[modId], depModId)
} else {
	m.Dependencies[modId] = []int{depModId}
}
Copy the code

That’s the end of the parsing.

Render the result of parsing

The gadget has one last step left to do, which is to render the parsed data to meet the graphviz tool’s requirements. The implementation code is the Render part:

First, define a template to generate the desired output format.

var graphTemplate = `digraph { node [shape=box]; {{ range $mod, $modId := .mods -}} {{ $modId }} [label="{{ $mod }}"]; {{ end -}} {{- range $modId, $depModIds := .dependencies -}} {{- range $_, $depModId := $depModIds -}} {{ $modId }} -> {{ $depModId }}; {{ end -}} {{- end -}} } `
Copy the code

There is nothing to be said for this section, but to familiarize yourself with the syntax of the TEXT /template template in Go. In order to be friendly, we remove the line break by – implementing it, without affecting the reading as a whole.

Next, look at the implementation of the Render method and put the previously parsed Mods and Dependencies into the template for rendering.

func (m *ModuleGraph) Render(w io.Writer) error {
	templ, err := template.New("graph").Parse(graphTemplate)
	iferr ! =nil {
		return fmt.Errorf("templ.Parse: %v", err)
	}

	if err := templ.Execute(w, map[string]interface{} {"mods":         m.Mods,
		"dependencies": m.Dependencies, }); err ! =nil {
		return fmt.Errorf("templ.Execute: %v", err)
	}

	return nil
}
Copy the code

Now all the work is done. Finally, integrate the process into the main function. The next step is to use it.

Experience with

Get started. In addition, I have only tested the use of this tool under Mac. If you have any questions, please feel free to ask.

First, install Graphviz, as described at the beginning of this article. Choose your system installation mode.

To install modv, run the following command:

$ go get github.com/poloxue/modv
Copy the code

Installation complete! A simple test of its use.

Take MacOS, for example. First download testing library, github.com/poloxue/testmod. Go to the testmod directory and run the following command:

$ go mod graph | modv | dot -T png | open -f -a /Applications/Preview.app
Copy the code

If the execution succeeds, the following results are displayed:

This perfectly demonstrates the dependencies between modules.

Some think

This is a hands-on article that goes from a simple idea to a successful presentation of a working tool. Although, the development is not difficult, from development to completion, only took an hour or two. But my sense is that this is a tool of real value.

Some ideas have not been implemented and validated, such as whether it is convenient to show the dependency tree of a given node rather than the entire project once the project is large. Also, whether the gadget will add value when other projects migrate to the Go Module.

---

Welcome to follow my wechat official account.