[frodo-v2.0] Embrace Golang, go~! layout: post date: 2020-06-15 tag: note author: BY Zhi-kai Yang

Frodo- V2.0 does not add new features, but reconstructs the most important part of the back end, the back end API using Golang, and Python is now only responsible for rendering the front end template. What was once a single-service application becomes a multi-service application. This article provides an overview of v2.0 tweaking and golang asynchro features. Please refer to the project address for the new version of the deployment documentation

The project address

Blog address

The main reconstructed modules are as follows:

  • Background CRUD interface for blog posts, users, tags, etc
  • Cache cleaning module
  • JWT certification module

why golang?

Golang is a young language, a static language of the new century. The goroutine design paradigm is designed to move away from multithreaded concurrency, balancing the strengths and limitations of dynamic languages such as C++ and javascript/python. The Web background and micro services are the most used areas of GO language, so I rewrite the pure API part of the background with GO.

I started to learn the GO language when I started to use Kubernetes in 2019. At that time, the project had a demand to expand an API of K8S. The official advice was that golang would be the best contributor for real contributor development. Go must be represented by Docker and Kubernetes, the most mainstream container and container orchestration tools.

Second, it’s not painful for me to use Go. Its style is between static and dynamic languages, so you use Pointers to avoid redundant object copying, and you can also use dynamic data structures like Python that are convenient. The advantage of class C is that it is not as close to the bottom as necessary to consider pointer manipulation and typing.

Python is also increasingly promoting display types. Type checking was introduced in V1.0, which does not provide performance gains in Python, but is good for debugging and docking static languages. Golang’s language therefore poses no difficulties.

Python with types is very similar to Golang:

async def get_user_by_id(id: int) -> User:
    user: User = await User.async_first(id)
    return user
Copy the code

In Golang types are mandatory:

func GetUsers(page int) (users []User){ DB.Where(...) .First(&user)return
}
Copy the code

In C++, it is obvious that different types are placed differently:

User* getUserById(int id) {
    user = User{id}
    User::first(&user)
    return user
}
Copy the code

In the end, what matters most is how Golang circles around it. Golang doesn’t have a lot of choices compared to Python-Web, but it does have enough. Gin and GORM are both lightweight and simple frameworks.

Challenge

The wheels of the golang

People who are used to writing dynamic reasons (especially Python/JS) will find Golang data structures troublesome:

  • map/structNew attributes cannot be added dynamically
  • There is noinThis frequently used feature
  • Any typeinterface{}Conversions to other types are not that simple
  • struct.map.jsonThe transition between them is not very natural
  • Be aware of value and address passing
  • There are no convenient set operations, such as union and difference, such as sorting, such as formatting and generation.
  • .

Fortunately, the open source community for Go is doing a great job. You can drink the packages that others have done on Github, and many wheels are already available. Go to https://godoc.org/ to search for officially supported wheels, which are generally stable and officially approved, and you can easily check their documentation. For set operations I use the library goset. If you do not find it in the official website, you can directly seek Github and reference the warehouse address. (Find the Golang package module convenient? So far, yes, but there are a lot of holes…)

Multi-service network deployment

Did not expect the V2.0 version is the most trouble in the deployment…

Golang and Uvicorn each have two ports. Adjust the static file accordingly, but since my deployment can only expose one port (due to domain name issues, see below), I can only use Nginx for forwarding.

The above structure has several configuration difficulties:

  • Static resource addressing and routing configuration. V1.0, but the language version is better configured to directly map local addresses. Now you need to explicitly configure forwarding in Nginx by service. Change the original local address to a domain name address for static resources.

  • Golang also calls Python services. For example, the “dynamic” API is still in Python, and the “post” API is in Golang, while the “post” API needs to be created after the “dynamic”, golang needs to call Python services. (This is perfectly normal; large projects need to communicate with each other.) Fortunately, this problem is much easier to solve on a single machine.

  • Wait, does caching conflict? In “Data”, Frodo has a caching mechanism. Now it is found that both Python’s foreground and Golang’s background rely on caching, which requires strict unification of keys to ensure the consistency of the two cached data.

Asynchrony and concurrency in Golang

Now that you’ve replaced the python service with Golang, does golang satisfy the asynchronous feature mentioned earlier? The idea is the same, but instead of asycio and awaitables, we’ve changed it to Goroutine.

func CreatePost(data map[string]interface{}) {
	post := new(Post)
	post.Title = data["title"]. (string)
	post.Summary = data["summary"]. (string)
	post.Type = data["type"]. (int)
	post.CanComment = data["can_comment"]. (int)
	post.AuthorID = data["author_id"]. (int)
	post.Status = data["status"]. (int)

	tags := data["tags"([]].string)
	content := data["content"]
	DB.Create(&post)

	fmt.Println(post)

	go post.SetProps("content", content) // go sets the content
	go post.UpdateTags(tags) // go updates the tag
	go post.Flush() // go clears the cache
	go CreateActivity(post) // go creates a dynamic
}
Copy the code

These are Goroutines, with communication tools and synchronization primitives that manage them. Each Goroutine can also continue to distribute coroutines, such as the update tag in it:

func (post *Post) UpdateTags(tagNames []string) {
	var originTags []Posttag
	var originTagNames []string

	DB.Where("post_id = ?", post.ID).Find(&originTags)
	for _, item := range originTags {
		var tag Tag
		DB.Select("name").Where("id = ?", item.TagID).First(&tag)
		originTagNames = append(originTagNames, tag.Name)
	}
	_, _, deleteTagNames, addTagNames := goset.Difference(originTagNames, tagNames)
	for _, tag := range addTagNames.([]string) {
		go CreateTags(tag)
		go CreatePostTags(post.ID, tag)
	}
	for _, tag := range deleteTagNames.([]string) {
		go DeletePostTags(post.ID, tag)
	}
}
Copy the code

Golang does not have a distribution like Asyncio. Gather (* Coros), and is implemented using a for loop.

Now that I’ve split the simple system into two services with different technology types, I can see the deployment challenges coming up. The next update is virtualization to solve the environment dependency problems and automated deployment