preface

Recently, I had to write Go for some time for the project. Compared to Java, it has simple syntax and some syntactic sugar like Python, which makes people shout “delicious”.

But at the moment, I still write more Python, and occasionally I have to go back to Java. Naturally, I am not familiar with Go.

So I used the weekend to do a small project to deepen some experience. So I came up with a blog widget I had written in Java.

During that time, a large number of pictures on Weibo were banned from being linked outside, making pictures in many personal blogs unviewable. This tool can back up the images in the article to the local, and can also directly replace the images to another map bed.

I personally have been using it all the time, usually in the code word using tools like iPic upload pictures to weibo map bed (mainly convenient + free). Once you’re done, use the tool to switch to a paid map bed such as [sm.ms](http://sm.MS) with one click and back up the image to your local disk.

After changing to Go to rewrite as cli tool, the effect is as follows:

What skills are needed

The reason why I chose this tool is to rewrite it with Go; One is that the function is relatively simple, but it can also take advantage of some characteristics of Go, such as network IO, coprogramming step and so on.

Does it feel more geeky to be a command-line tool?

Before we get started, let’s introduce some general knowledge points for javaers who are not familiar with Go:

  • Use and manage third-party dependencies (go mod)
  • Use of coroutines.
  • Multi-platform packaging.

Let’s start with the specific operation. I think even those friends who have little contact with Go can quickly get started and realize a small tool after watching it.

Use and manage third-party dependencies

  • If you have not installed Go, please refer to the official website to install it yourself.

First, let’s take a look at dependency management in Go. The dependency management module was officially shipped after version 1.11, so it has been strongly recommended in the latest version 1.15.

It is similar in purpose and function to Maven in Java and PIP in Python, but much simpler to use than Maven.

According to its instructions, first execute go mod init in the project directory to initialize a go.mod file. Of course, if you are using an IDE like GoLang, the directory structure will be created automatically when you create a new project, including the go.mod file.

In this file we import the third-party packages we need:

module btb

go 1.15

require (
	github.com/cheggaaa/pb/v3 v3. 0. 5
	github.com/fatih/color v110.. 0
	github.com/urfave/cli/v2 v23.. 0
)
Copy the code

I used three packages here, which are:

  • pb: Progress bar, used to output the progress bar on the console.
  • color: Used to output text in different colors on the console.
  • cli: Command line tool development package.
import (
	"btb/constants"
	"btb/service"
	"github.com/urfave/cli/v2"
	"log"
	"os"
)

func main(a) {
	var model string
	downloadPath := constants.DownloadPath
	markdownPath := constants.MarkdownPath

	app := &cli.App{
		Flags: []cli.Flag{
			&cli.StringFlag{
				Name:        "model",
				Usage:       "operating mode; r:replace, b:backup",
				DefaultText: "b",
				Aliases:     []string{"m"},
				Required:    true,
				Destination: &model,
			},
			&cli.StringFlag{
				Name:        "download-path",
				Usage:       "The path where the image is stored",
				Aliases:     []string{"dp"},
				Destination: &downloadPath,
				Required:    true,
				Value:       constants.DownloadPath,
			},
			&cli.StringFlag{
				Name:        "markdown-path",
				Usage:       "The path where the markdown file is stored",
				Aliases:     []string{"mp"},
				Destination: &markdownPath,
				Required:    true,
				Value:       constants.MarkdownPath,
			},
		},
		Action: func(c *cli.Context) error {
			service.DownLoadPic(markdownPath, downloadPath)

			return nil
		},
		Name:  "btb",
		Usage: "Help you backup and replace your blog's images",
	}

	err := app.Run(os.Args)
	iferr ! =nil {
		log.Fatal(err)
	}
}
Copy the code

The code is very simple. It simply uses the API provided by the CLI to create several commands that map the -dp and -mp parameters entered by the user to downloadPath and markdownPath variables.

This data is then used to scan all images and download them to the appropriate directory.

For more instructions, please refer to the official documentation.

You can see that some of the syntax is completely different from Java, for example:

  • When declaring a variable, the type is put behind, and the variable name is defined first. The method parameters are similar.
  • Type derivation, variable types may not be specified (new versionJavaAlso support)
  • Method supports simultaneous return of multiple values, which is handy.
  • Public and private functions are distinguished by case.
  • I don’t want to list all the others.

coroutines

DownLoadPic(markdownPath, downloadPath) is invoked to process the service logic.

File scanning, image downloading and other code included here will not be analyzed; The official SDK is clear and simple.

Focus on the Goroutime in Go which is the coroutine.

The scenario I use here is to use a coroutine to parse and download images every time a file is scanned, thus improving the overall efficiency of the operation.

func DownLoadPic(markdownPath, downloadPath string) {
	wg := sync.WaitGroup{}
	allFile, err := util.GetAllFile(markdownPath)
	wg.Add(len(*allFile))

	iferr ! =nil {
		log.Fatal("read file error")}for _, filePath := range *allFile {

		go func(filePath string) {
			allLine, err := util.ReadFileLine(filePath)
			iferr ! =nil {
				log.Fatal(err)
			}
			availableImgs := util.MatchAvailableImg(allLine)
			bar := pb.ProgressBarTemplate(constants.PbTmpl).Start(len(*availableImgs))
			bar.Set("fileName", filePath).
				SetWidth(120)

			for _, url := range *availableImgs {
				iferr ! =nil {
					log.Fatal(err)
				}
				err := util.DownloadFile(url, *genFullFileName(downloadPath, filePath, &url))
				iferr ! =nil {
					log.Fatal(err)
				}
				bar.Increment()

			}
			bar.Finish()
			wg.Done()

		}(filePath)
	}
	wg.Wait()
	color.Green("Successful handling of [%v] files.\n".len(*allFile))

	iferr ! =nil {
		log.Fatal(err)
	}
}
Copy the code

We don’t need to maintain an executorService like Java does, and we don’t need to worry about the size of the thread pool. Go does all the scheduling itself.

You only need to add the go keyword before calling a function, but this is an anonymous function.

And because GorouTime is very lightweight and takes up much less memory than Threads in Java, we don’t need to control exactly how many threads are created.

But there is also something very similar to Java: WaitGroup.

Its usage and function are very similar to CountDownLatch in Java; It is used to wait for all goroutime to complete, in this case it is used to wait for all images to download and then exit the program.

It is mainly divided into three steps:

  • Create and initializegoruntimeThe number of:wg.Add(len(number)
  • Whenever agoruntimeCall donewg.Done()Let’s decrease the count by one.
  • The final callwg.Wait()Waiting for theWaitGroupThe number of alpha is reduced to 0.

For coroutine Go, it is recommended to use Chanel to communicate with each other, which will be discussed later.

packaging

So much for the core logic, packaging and running; This is a big difference from Java.

As we all know, Java has a saying: Write once run anywhere

This is because with the JVM, we only need to print out one package regardless of which platform the code ends up running on; But how does Go run on all platforms without a virtual machine?

Simply put, Go can be packaged into a different binary file for different platforms, which contains all the dependencies needed to run, without even having to install the Go environment on the target platform.

  • Although Java will ultimately only need one package, it will also need to be compatible across platformsJavaRun environment.

I wrote a Makefile here to perform the packaging: make Release

# Binary name
BINARY=btb
GOBUILD=go build -ldflags "-s -w"-o ${BINARY} GOCLEAN=go clean RMTARGZ=rm -rf *. Gz VERSION=0.0.1release:
	# Clean
	$(GOCLEAN)
	$(RMTARGZ)
	# Build for mac
	CGO_ENABLED=0 GOOS=darwin GOARCH=amd64 $(GOBUILD)
	tar czvf ${BINARY}-mac64-${VERSION}.tar.gz ./${BINARY}
	# Build for arm
	$(GOCLEAN)
	CGO_ENABLED=0 GOOS=linux GOARCH=arm64 $(GOBUILD)
	tar czvf ${BINARY}-arm64-${VERSION}.tar.gz ./${BINARY}
	# Build for linux
	$(GOCLEAN)
	CGO_ENABLED=0 GOOS=linux GOARCH=amd64 $(GOBUILD)
	tar czvf ${BINARY}-linux64-${VERSION}.tar.gz ./${BINARY}
	# Build for win
	$(GOCLEAN)
	CGO_ENABLED=0 GOOS=windows GOARCH=amd64 $(GOBUILD).exe
	tar czvf ${BINARY}-win64-${VERSION}.tar.gz ./${BINARY}.exe
	$(GOCLEAN)
Copy the code

As you can see, we only need to specify system variables before go build to print packages for different platforms. For example, we packaged the arm64 architecture for Linux:

CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build main.go -o btb

You can run the./ BTB program directly on the target platform.

conclusion

All code for this article has been uploaded to Github: github.com/crossoverJi…

If you are interested, you can run the installation script to experience it.

curl -fsSL https://raw.githubusercontent.com/crossoverJie/btb/master/install.sh | bash
Copy the code
  • At present this version only realizes the picture download backup, will improve the chart bed replacement and other functions later.

During this period of time, I was deeply impressed by Go. For Java at the age of 25, Go is indeed a terrible afterthought, and what is more irritating is that it caught up with the wave of cloud native.

Some minor problems that seemed unimportant before have been highlighted, such as slow startup, large memory footprint, verbose syntax, etc. Still, I’m looking forward to seeing positive changes in the new version of Java, not to mention its unshaken ecosystem.

For more Java follow-up content, please refer to Zhou Zhiming’s article: In the era of cloud native, Java is in danger?