Odd technical guidelines

Application programming class last section, this class teacher Li Gang shared a lot of his Go language development used in the tools and some practical experience and experience, let us feel together to learn this final work!

This article is reprinted from HULK First Line Technology Talk.

The International Space Station, Oct. 4, 2018

by NASA IOTD

Today I’m going to show you some programming practices I’ve done with Go.

A, golog

The address of the project: https://github.com/goinbox/golog

Log is a strong requirement for any development we do, so let me tell you a little bit about golog.

Let’s take a look at the relationships among the most important data structures:

  • writer

To encapsulate underlying write operations, write objects can be files, queues, or ES. The following writer implementation is provided:

  1. FileWriter write files

  2. FileWithSplitWriter Writes files automatically by day or hour

  3. Write ConsoleWriter terminal

  4. buffer

Adding buffer to write operations to improve write performance is implemented in the following ways:

  1. Implement as writer decorator

  2. Provide separate Goroutine to do autoflush

  3. formater

Formater is used to format the log content to be recorded before sending it to writer, for example, adding a unified log date or adding a color to terminal output. The current implementation is as follows:

  1. SimpleFormater prefixes the message with logLevel and time

  2. WebFormater adds clientIp and logId to simpleFormater

  3. ConsoleFormater adds color to terminal output

  4. logger

The simpleLogger implementation is currently provided as a synchronous method (write blocking).

  • async

Putting writes into a separate Goroutine improves program performance by:

  1. AsyncLogger is implemented as a decorator to logger

  2. Provides a separate Goroutine for write operations

For more detailed usage, please refer to:

https://www.jianshu.com/p/20d0f74c3c08

Second, the shardmap

The address of the project: https://github.com/goinbox/shardmap

To improve performance, the core idea is to reduce the lock granularity. Shardmap is developed in this way:

Sync. Map is provided in the official package after GO1.8 to solve the problem of concurrent read and write of Map, but my own test performance is not as good as shardMap, readers can try it for themselves.

For more detailed usage, please refer to:

https://www.jianshu.com/p/090e00f12b3e

Third, redis

The address of the project: https://github.com/goinbox/redis

There are a lot of packages available in Redis. I implemented this package myself, and the underlying driver part uses Redigo. Considering the actual production environment, I implemented the following mechanism by myself:

  1. Lazy loading mechanism, where network connections are created only when you actually interact with Redis

  2. Automatic reconnection mechanism for operation failure

  3. Provide connection pooling to improve performance

  4. Pipeling encapsulation

  5. Things to encapsulate

For more detailed usage, please refer to:

https://www.jianshu.com/p/fb498f30dff2

Four, goconsumer

The address of the project: https://github.com/goinbox/goconsumer

The use of asynchronous queues is also currently essential in development, and a consumption processing framework is provided that currently supports:

  1. Message out-of-order consumption

  2. Message order consumption

The overall processing framework is shown as follows:

The object relationships are as follows:

  • consumer

Objects that consume from various queues, such as Kafka, NSQ, etc

  • dispatcher

To allocate messages to worker for processing, we can implement our own allocation algorithm here to achieve the purpose of sequential consumption. Currently, the following two implementations are provided:

  1. SimpleDispatcher, this does random distribution of messages and can be used without sequential consumption

  2. SpecifyDispatcher, for instance, can be used for special consumer needs such as sequential consumption by specifying message distribution methods themselves

  3. worker

Message processing object that does the actual business work.

  • task

Start a consumption task framework with an entry point for execution. I have a demo implementation in task_test.go for your reference.

Fifth, gobox – demo

The address of the project: https://github.com/goinbox/gobox-demo

Gobox-demo is a general template project developed by me. Based on this project, I would like to introduce some experience of using Go language to do project development to you.

Organization of controllers and actions

Most projects I’ve seen like to put Controlelr and Action in a single code file, and the more features the project has, the longer the file will be.

In practice this would be a major development and maintenance disadvantage, so I broke them down and made each action a code file so it was clear:

The organization of the SVC

other

These are some of the tools I use most often during development. My rule has always been to keep things lean and not add features that the team doesn’t use.

In addition, some code related to internal company, such as on-line operation can not be shown here, but there will be a separate directory to organize them.

In a word, I strive to make the project reasonably organized, with a clear name and clear hierarchy, hoping that people can judge which part of function is put where from the organizational structure of the project, and improve any ambiguity.

conclusion

Application programming class has all finished, I hope my experience is helpful to you, there is improper understanding of the place also please correct, thank you!

World of you when not

Just your shoulder

There is no


360 official technical official account

Technology of dry goods | | hand information activities

Empty,