Go Micro-service framework GO-Micro Deep Learning (I) Introduction to the overall architecture

In addition to the stability of the microservitization project, I am also concerned about the following issues:

I. Efficiency and security of data transmission between services.

Two: dynamic expansion of services, that is, registration and discovery of services, service clustering.

Three: customization of micro-service functions, because not all functions will meet your needs, it is inevitable to develop some functions according to their own needs.

Go-micro is a very good RPC microservice framework based on GO language, with perfect functions, and several problems I care about have been solved well:

A: service transmission format for protobuf, efficiency is very easy to say, very fast, also very safe.

Two: Go-Micro’s service registration and discovery is diverse. I personally prefer etcDV3 services service discovery and registration.

Three: the main functions have corresponding interfaces, as long as the corresponding interface, you can according to their own needs customized plug-in.

In my spare time, I read the source code of Go-Micro systematically. The more I read, the more I felt that this framework was written well and learned a lot from it. I just want to organize a series of posts to share my experience of learning Go-Micro with you.

The communication process

The go-Micro communication flow is as follows

Server listens to the call of the client and processes the information pushed by Brocker. In addition, the Server needs to Register its existence or extinction with the Register so that the Client can know its status.

Register service registration discovery.

The Client obtains the Server information from the Register, and then selects a Server according to the algorithm for communication each time. Of course, the communication needs to go through a series of processes such as encoding/decoding and selecting transmission protocol.

If there is a need to notify all Server end can use Brocker to push information.

Brocker information queue receives and publishes information.

Go-micro can be highly customized because of its framework structure. Go-micro consists of eight key interfaces, each of which can be reimplemented according to its own requirements. These eight primary intefaces also form the framework of Go-Micro.

Each of these interfaces, Go-micir, has its own default implementation, and a Go-plugins is an alternative to the implementation of these interfaces. You can also implement your own plug-ins on demand

This post mainly introduces the main structure of go-Micro and the functions of these interfaces. We will talk about the details in future articles:

Transort

An interface for communication between services. The ultimate implementation of how services are sent and received is customized by these interfaces.

Source:

type Socket interface {
    Recv(*Message) error
    Send(*Message) error
    Close() error
}

type Client interface {
    Socket
}

type Listener interface {
    Addr() string
    Close() error
    Accept(func(Socket)) error
}

typeTransport interface { Dial(addr string, opts ... DialOption) (Client, error) Listen(addr string, opts ... ListenOption) (Listener, error) String() string }Copy the code

Transport’s Listen method is usually called by the Server. It listens on a port and waits for the client to call.

The Dial of Transport is the way a client connects to a service. It returns a Client interface, which returns a Client interface embedded in the Socket interface, whose method is to send and receive communications.

HTTP transport is go-Micro’s default synchronous communication mechanism. Of course there are many other plug-ins: GRPC, nats, TCP, udp, the rabbitmq, nats, are currently has achieved. You can find them all in Go-plugins.

Codec

With transport, the next step is to solve the transport encoding and decoding problem. Go-micro has many encoding and decoding methods, the default implementation is Protobuf, of course there are other implementations, json, Protobuf, JSONrpc, Mercury, etc.

The source code

type Codec interface {
    ReadHeader(*Message, MessageType) error
    ReadBody(interface{}) error
    Write(*Message, interface{}) error
    Close() error
    String() string
}

type Message struct {
    Id     uint64
    Type   MessageType
    Target string
    Method string
    Error  string
    Header map[string]string
}Copy the code

The Write method of the Codec interface is the encoding process, and the two reads are the decoding process.

Registry

Service registration and discovery, the current implementation of consul, MDNS, etcd, etcdv3, zookeeper, kubernetes. And so on,

typeRegistry interface { Register(*Service, ... RegisterOption) error Deregister(*Service) error GetService(string) ([]*Service, error) ListServices() ([]*Service, error) Watch(... WatchOption) (Watcher, error) String() string Options() Options }Copy the code

The Client uses the watch method to monitor the Service. This method will be triggered when a Service is added or deleted to remind the Client to update the Service information.

The default for service registration and discovery is Consul, but it is not recommended for individuals as you cannot use Consul clusters directly

I personally prefer the ETCDV3 cluster. You can choose according to your own preference.

Selector

Based on Registry, the Selector is a load balancer at the client level. When a client sends a request to a Service, the Selector obtains the available Service node from the host list in Registery according to different algorithms and communicates with it. The current implementation of cyclic algorithm and random algorithm, the default is random algorithm.

Source:

typeSelector interface { Init(opts ... Option) error Options() Options // Select returns afunction which should returnthe next node Select(service string, opts ... SelectOption) (Next, error) // Mark sets the success/error against a node Mark(service string, node *registry.Node, err error) // Reset returns state back to zerofor a service
    Reset(service string)
    // Close renders the selector unusable
    Close() error
    // Name of the selector
    String() string
}Copy the code

The default implementation is local cache, the current implementation of blacklist,label,named and so on.

Broker

The Broker is the interface to publish and subscribe messages. A very simple example, because the node of the service is not fixed, if there is a need to modify all the service behavior, you can make the service subscribe to a topic, when there is information published, all the listening service will receive the information, according to your need to do the corresponding behavior.

The source code

typeBroker interface { Options() Options Address() string Connect() error Disconnect() error Init(... Option) error Publish(string, *Message, ... PublishOption) error Subscribe(string, Handler, ... SubscribeOption) (Subscriber, error) String() string }Copy the code

The default implementation of the Broker is HTTP, but this should not be used in a production environment. There are many mature message queue implementations in Go-plugins, such as Kafka, NSQ, RabbitMQ, Redis, and so on.

Client

Client is the interface that requests the service. It encapsulates Transport and Codec for RPC calls, and also encapsulates Brocker for information publishing.

The source code

typeClient interface { Init(... Option) error Options() Options NewMessage(topic string, msg interface{}, opts ... MessageOption) Message NewRequest(service, method string, req interface{}, reqOpts ... RequestOption) Request Call(ctx context.Context, req Request, rsp interface{}, opts ... CallOption) error Stream(ctx context.Context, req Request, opts ... CallOption) (Stream, error) Publish(ctx context.Context, msg Message, opts ... PublishOption) error String() string }Copy the code

Of course, it also supports the implementation and use of duplex communication Stream, which will be explained in detail later.

The default implementation is RPC, GRPC and HTTP, which can be found in Go-plugins

Server

Server is a name that you know what it does. Listen for RPC requests. Listen for broker subscriptions, wait for messages to be pushed from queues, and so on.

The source code

typeServer interface { Options() Options Init(... Option) error Handle(Handler) error NewHandler(interface{}, ... HandlerOption) Handler NewSubscriber(string, interface{}, ... SubscriberOption) Subscriber Subscribe(Subscriber) error Register() error Deregister() error Start() error Stop() error String() string }Copy the code

The default implementation is RPC, GRPC and HTTP, which can be found in Go-plugins

Service

Service is the encapsulation of Client and Server. It contains a series of methods to initialize Service and Client using initial values, making it easy to create an RPC Service.

Source:

typeService interface { Init(... Option) Options() Options Client() client.Client Server() server.Server Run() error String() string }Copy the code

I will give you the details one by one in future posts. I hope this post can help you get a preliminary understanding of the overall framework of Go-Micro