This is the 12th day of my participation in the August More Text Challenge. For details, see: August More Text Challenge.
In the Web system, there are four ways for the front end to pass parameters to the back end through HTTP request. The parameters can be placed in the request path, Query parameter, HTTP header, and HTTP protocol body. The parameters in the protocol body have many formats, such as JSON, form, and so on. Of course, the front-end may also choose other protocols, such as Websocket, GRPC-Web, and so on, the specific parameter form is different from HTTP.
Faced with such a variety of technical implementation, when we want to design a Web system, how to choose? What kind of norms should be followed? That’s what I’m going to tell you about.
What problems does RESTful solve?
First, let’s look at three time points:
- 1991 HTTP 0.9 was born
- In May 1996, HTTP 1.0 was born
- In January 1997, HTTP 1.1 was born
Before THE advent of HTTP 1.0, that is, the ERA of HTTP 0.9, the HTTP protocol only supported GET requests, and all parameters could only be passed through the URL. For example, a request to GET a dynamic web page might look like GET /index.php, right? Page = hello&user = 1234 & action = the view.
With the advent of HTTP 1.0, there were POST and HEAD requests that supported passing parameters from the HTTP header and body. But HTTP 1.0 didn’t solve the problem of having to establish a new connection with every request, and THEN HTTP 1.1 came along pretty quickly.
In addition to maintaining connections, HTTP 1.1 adds a variety of new methods, such as PUT, DELETE, PATCH, and OPTIONS. While it’s more powerful and richer, it raises a new question: how do you choose these methods? For example, if I want to upload data, should I use POST, PUT or PATCH? This certainly increases the cost of choice.
Then comes REST (Representational State Transfer), which is simply a set of architectural constraints and principles, and design that complies with REST rules is RESTful.
Because THE HTTP protocol itself is stateless, but the back-end data is stateful, it is challenging to manipulate stateful data using stateless protocols. For example, when you only use GET requests, you can’t intuitively tell from the request parameters what to do with the data, because there is no uniform name for GET request parameters. In the previous example, action=view might also be defined as a=v.
How does RESTful solve this problem? It makes full use of the characteristics of various HTTP request methods to express the specific operation mode of data. For example, when you want to add data, you should use the POST method. To replace the entire data, use the PUT method; When only partial fields of data are replaced, the PATCH method should be used. If you are deleting data, the DELETE method should be used. In this way, it is clear what to do with the data.
What problem does RPC solve?
The full name of RPC is Remote Procedure Call. It, like REST, is a technique by which a client makes a request to a server.
I mentioned earlier that RESTful solves many problems, but why do YOU need RPC?
First, as mentioned earlier, the HTTP protocol has added a lot of functionality since HTTP 1.0. Before the advent of HTTP2, THE CONTENT of THE HTTP protocol was in text format, but the content of the HTTP protocol became more and more bloated as more and more functions were added. In some high performance scenarios, especially when frequent requests are made between internal services of the system, the bloated protocol will bring a lot of network overhead, which affects the service performance. However, in the underlying protocol of RPC, you can directly use TCP as a long connection, and the protocol content can be simplified. RPC can also compress data before transmission, which is much better than HTTP.
Second, RESTful is a convention, not a mandatory norm, and you may or may not follow it. This leads to inconsistent interface styles and behaviors among people of different skill levels.
For example, a novice programmer implementing an API using HTTP may accidentally return extra data to the client. The client may not report an error, but there is a risk of data leakage. RPC typically uses an Interactive Data Language (IDL) like Protobuf to define interface specifications and generate framework code on both the client and server sides.
Among them, THE function of IDL is to define the data interaction mode between the client and the server, and use the generated framework code to form strong constraints when the two sides develop the code, which is a kind of embodiment of contract programming. When using RPC, the server can only return data defined in IDL, otherwise the framework code will report an error.
Currently, Thrift and gRPC are the main cross-platform RPC protocols. Thrift is open source by Facebook and transmits data primarily over TCP. GRPC is open source by Google, the underlying HTTP2 protocol based. Because gRPC supports rich middleware and two-way flow mode, it has strong ease of use and has been applied more and more widely. For example, ETCD provides a gRPC interface.
Seckill API design
** Seckill system API has two main blocks: API service, Admin service. ** Next, I’ll introduce you to RESTful and gRPC in these two services.
API Service API design
According to our previous demand analysis, seckill API service mainly provides users with activity information and the function of snapping up products. Then, the APIS of seckill API service should be divided into two groups: event information and panic buying, which are represented by Event and shop respectively.
In the Event function, we need to provide three interfaces to the front end:
-
The first is /event/list, which uses the GET method to GET a list of all ongoing or upcoming activities.
-
The second is /event/info, using the GET method, used to query the current seckill activity information of a commodity, the parameter is the commodity ID;
-
The third is /event/subscribe, using the POST method, used to subscribe to a commodity activity start notification, parameters are the event ID, commodity ID, device ID.
In the shop function, we only need to provide a snapping interface: /shop/cart/add, specific can use the PUT method, that is, to add the product to the shopping cart, the parameter is the product ID. The PUT method is used here because the user’s shopping cart is always there, and the PUT method is RESTful.
These interfaces all return the same information, such as error number, message, whether the user has logged in, data, and so on. They can be defined using Go constructs, as follows:
Type Response struct {Code int 'json:" Code "' // Service error Code Data interface{} 'JSON :" Data "' // Data Msg string 'JSON :" Msg "' // Message}Copy the codeNote that in order to support returning multiple types of Data, you need to define the Data field in the structure to be of type Interface. In addition, the Login Status login-status is returned to the front end in the HTTP Header. In order to let the admin can also reuse the structure definition, we need to put it in infrastructure/utils/response. Go.
Next, we define the Event and Shop applications in the Application/API, and define the handler functions corresponding to the previous interfaces as follows:
type Event struct{
type Shop struct{
func (e *Event) List(ctx *gin.Context) {
resp := &utils.Response{
Code: 0,
Data: nil,
Msg: "ok",
}
status := http.StatusOK
logrus.Info("event list")
ctx.JSON(status, resp)
}
func (e *Event) Info(ctx *gin.Context) {
resp := &utils.Response{
Code: 0,
Data: nil,
Msg: "ok",
}
status := http.StatusOK
logrus.Info("event info")
ctx.JSON(status, resp)
}
func (e *Event) Subscribe(ctx *gin.Context) {
resp := &utils.Response{
Code: 0,
Data: nil,
Msg: "ok",
}
status := http.StatusOK
logrus.Info("event subscribe")
ctx.JSON(status, resp)
}
func (s *Shop) AddCart(ctx *gin.Context) {
resp := &utils.Response{
Code: 0,
Data: nil,
Msg: "ok",
}
status := http.StatusOK
logrus.Info("shop add cart")
ctx.JSON(status, resp)
}
Copy the codeNext, we implement an initRouters function that registers the interface functions implemented earlier into the routing of the Gin framework. The code is in the interfaces/ API directory for these routers. Go file, as shown below:
func initRouters(g *gin.Engine) {
logrus.Info("init api routers")
event := g.Group("/event")
eventApp := api.Event{}
event.GET("/list", eventApp.List)
event.GET("/info", eventApp.Info)
event.POST("/subscribe", eventApp.Subscribe)
shop := g.Group("/shop")
shopApp := api.Shop{}
shop.PUT("/cart/add", shopApp.AddCart)
}
Copy the codeWe can then call the initRouters function in the Run function of the APi. go file to register routes to the Gin framework.
In addition to implementing the API for browser requests, we also need to implement an RPC service for admin requests to synchronize the active configuration, providing the online and offline functions for topics and sessions. Application/API/RPC directory event.proto, defined as:
syntax = "proto3";
package rpc;
message Goods {
int32 id = 1;
string desc = 2;
string img = 3;
string price = 4;
string event_price = 5;
int32 event_stock = 6;
}
message Event {
int32 id = 1;
int32 topic_id = 2;
int64 start_time = 3;
int64 end_time = 4;
int32 limit = 5;
repeated Goods goods_list = 6;
}
message Topic {
int32 id = 1;
string title = 2;
string desc = 3;
string banner = 4;
int64 start_time = 5;
int64 end_time = 6;
}
message Response {
int32 code = 1;
string msg = 2;
}
service EventRPC {
rpc EventOnline(Event) returns(Response);
rpc EventOffline(Event) returns(Response);
rpc TopicOnline(Topic) returns(Response);
rpc TopicOffline(Topic) returns(Response);
}
Copy the codeAs you can see, I’ve defined an RPC service called EventRPC in the Protobuf file, along with the parameters and return value types that will later generate the Go code for the server and client. How are we going to use this Protobuf file? Let’s add the command proto to the Makefile to compile protobuf, as follows:
proto: application/api/rpc/event.proto
protoc --go_out=plugins=grpc:./ application/api/rpc/event.proto
Copy the codeBy executing make proto, we can compile an event.pb.go file in the application/ API/RPC directory. In this file is the Go definition of the RPC interface, including the server and client definitions, which is the contract for the Admin service to communicate with the API service. The specific definition is as follows:
// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://godoc.org/google.golang.org/grpc#ClientConn.NewStream. type EventRPCClient interface { EventOnline(ctx context.Context, in *Event, opts ... grpc.CallOption) (*Response, error) EventOffline(ctx context.Context, in *Event, opts ... grpc.CallOption) (*Response, error) TopicOnline(ctx context.Context, in *Topic, opts ... grpc.CallOption) (*Response, error) TopicOffline(ctx context.Context, in *Topic, opts ... grpc.CallOption) (*Response, error) } // EventRPCServer is the server API for EventRPC service. type EventRPCServer interface { EventOnline(context.Context, *Event) (*Response, error) EventOffline(context.Context, *Event) (*Response, error) TopicOnline(context.Context, *Topic) (*Response, error) TopicOffline(context.Context, *Topic) (*Response, error) }Copy the codeThen, we need to implement the RPC service as defined by Event.pB.go. In application/ API/RPC. Go, the code is as follows:
type EventRPCServer struct {
}
func (s *EventRPCServer) EventOnline(ctx context.Context, evt *rpc.Event) (*rpc.Response, error) {
logrus.Info("event online ", evt)
resp := &rpc.Response{}
return resp, nil
}
func (s *EventRPCServer) EventOffline(ctx context.Context, evt *rpc.Event) (*rpc.Response, error) {
logrus.Info("event offline ", evt)
resp := &rpc.Response{}
return resp, nil
}
func (s *EventRPCServer) TopicOnline(ctx context.Context, t *rpc.Topic) (*rpc.Response, error) {
logrus.Info("topic online ", t)
resp := &rpc.Response{}
return resp, nil
}
func (s *EventRPCServer) TopicOffline(ctx context.Context, t *rpc.Topic) (*rpc.Response, error) {
logrus.Info("topic offline ", t)
resp := &rpc.Response{}
return resp, nil
}
Copy the codeSimilarly, we will provide startup and exit functions like API services, in the RPC. Go file in the directory interfaces/ RPC, as shown below:
var grpcS *grpc.Server func Run() error { bind := viper.GetString("api.rpc") logrus.Info("run RPC server on ", bind) lis, err := utils.Listen("tcp", bind) if err ! = nil { return err } grpcS = grpc.NewServer() eventRPC := &api.EventRPCServer{} rpc.RegisterEventRPCServer(grpcS, EventRPC) // Supports gRPC Reflection, Register(grpcS) return grpcs.serve (lis)} func Exit() {grpcs.gracefulstop () logrus.info (" RPC server exit") }Copy the codeNext, we need to add the code to start the RPC service in the start command:
package cmd import ( "os" "os/signal" "sync" "syscall" "github.com/letian0805/seckill/interfaces/rpc" "github.com/letian0805/seckill/interfaces/api" "github.com/sirupsen/logrus" "github.com/spf13/cobra" ) // apiCmd represents the api command var apiCmd = &cobra.Command{ Use: "api", Short: "Seckill api server.", Long: `Seckill api server.`, Run: func(cmd *cobra.Command, args []string) { wg := &sync.WaitGroup{} wg.Add(2) onApiExit := make(chan error, 1) onRpcExit := make(chan error, 1) go func() { defer wg.Done() if err := api.Run(); err ! = nil { logrus.Error(err) onApiExit <- err } close(onApiExit) }() go func() { defer wg.Done() if err := rpc.Run(); err ! = nil { logrus.Error(err) onRpcExit <- err } close(onRpcExit) }() onSignal := make(chan os.Signal) signal.Notify(onSignal, syscall.SIGINT, syscall.SIGTERM) select { case sig := <-onSignal: logrus.Info("exit by signal ", sig) api.Exit() rpc.Exit() case err := <-onApiExit: rpc.Exit() logrus.Info("exit by error ", err) case err := <-onRpcExit: api.Exit() logrus.Info("exit by error ", err) } wg.Wait() }, } func init() { rootCmd.AddCommand(apiCmd) }Copy the codeFinally, we compile the executable and run it.
Admin service API design
First of all, we review the functional requirements of the management background introduced in the demand analysis, which mainly includes the following sections: topic management, site management and commodity management. Each module has: list, create, view, modify, delete and other functions. Among them, special topics and sessions have online, offline these two functions. So, the list of interfaces we need to implement looks like this:
GET/POST /topic? Page =1&size=10 # GET /topic/{id} # GET /topic/{id} # GET /topic/{id} # GET /topic/{id} # GET /topic/{id} # GET /topic/{id} # GET /topic/{id /topic/{id} # POST /event # GET /eventpage=1&size=10 # GET /event/{id} # PUT /event/{id} # GET /event/{id} # PUT /event/{id} # PUT /event/{id}/{status} # DELETE /event/{id} # DELETE /event/{id} # POST /goods # GET /goods? Page =1&size=10 # DELETE /goods/{id} # DELETE /goodsCopy the codeDue to the large number of interfaces, we created three files in the application/admin directory, namely topic.go, Event. go and Goods. go, which are used to implement the API code of admin.
The topic application is defined in topic.go. It contains four methods: Post, Get, Put, and Delete.
type Topic struct{
func (t *Topic) Post(ctx *gin.Context) {
resp := &utils.Response{
Code: 0,
Data: nil,
Msg: "ok",
}
status := http.StatusOK
logrus.Info("topic post")
ctx.JSON(status, resp)
}
func (t *Topic) Get(ctx *gin.Context) {
resp := &utils.Response{
Code: 0,
Data: nil,
Msg: "ok",
}
status := http.StatusOK
logrus.Info("topic get")
ctx.JSON(status, resp)
}
func (t *Topic) Put(ctx *gin.Context) {
resp := &utils.Response{
Code: 0,
Data: nil,
Msg: "ok",
}
status := http.StatusOK
logrus.Info("topic put")
ctx.JSON(status, resp)
}
func (t *Topic) Delete(ctx *gin.Context) {
resp := &utils.Response{
Code: 0,
Data: nil,
Msg: "ok",
}
status := http.StatusOK
logrus.Info("topic delete")
ctx.JSON(status, resp)
}
Copy the codeGo defines the event application. Its four methods are named the same as topic. The code is as follows:
type Event struct{}
func (t *Event) Post(ctx *gin.Context) {
resp := &utils.Response{
Code: 0,
Data: nil,
Msg: "ok",
}
status := http.StatusOK
logrus.Info("event post")
ctx.JSON(status, resp)
}
func (t *Event) Get(ctx *gin.Context) {
resp := &utils.Response{
Code: 0,
Data: nil,
Msg: "ok",
}
status := http.StatusOK
logrus.Info("event get")
ctx.JSON(status, resp)
}
func (t *Event) Put(ctx *gin.Context) {
resp := &utils.Response{
Code: 0,
Data: nil,
Msg: "ok",
}
status := http.StatusOK
logrus.Info("event put")
ctx.JSON(status, resp)
}
func (t *Event) Delete(ctx *gin.Context) {
resp := &utils.Response{
Code: 0,
Data: nil,
Msg: "ok",
}
status := http.StatusOK
logrus.Info("event delete")
ctx.JSON(status, resp)
}
Copy the codeThe goods application is defined in goods.go.
type Goods struct{}
func (t *Goods) Post(ctx *gin.Context) {
resp := &utils.Response{
Code: 0,
Data: nil,
Msg: "ok",
}
status := http.StatusOK
logrus.Info("goods post")
ctx.JSON(status, resp)
}
func (t *Goods) Get(ctx *gin.Context) {
resp := &utils.Response{
Code: 0,
Data: nil,
Msg: "ok",
}
status := http.StatusOK
logrus.Info("goods get")
ctx.JSON(status, resp)
}
func (t *Goods) Put(ctx *gin.Context) {
resp := &utils.Response{
Code: 0,
Data: nil,
Msg: "ok",
}
status := http.StatusOK
logrus.Info("goods put")
ctx.JSON(status, resp)
}
func (t *Goods) Delete(ctx *gin.Context) {
resp := &utils.Response{
Code: 0,
Data: nil,
Msg: "ok",
}
status := http.StatusOK
logrus.Info("goods delete")
ctx.JSON(status, resp)
}
Copy the codeNext, we need to refer to the previous API command implementation, inject application associated with the routing and framework, namely in the interfaces/admin/routers. The implementation in the go initRouters function. The code is as follows:
func initRouters(g *gin.Engine) {
topic := g.Group("/topic")
topicApp := admin.Topic{}
topic.POST("/", topicApp.Post)
topic.GET("/", topicApp.Get)
topic.GET("/:id", topicApp.Get)
topic.PUT("/:id", topicApp.Put)
topic.PUT("/:id/:status", topicApp.Put)
topic.DELETE("/:id", topicApp.Delete)
event := g.Group("/event")
eventApp := admin.Event{}
event.POST("/", eventApp.Post)
event.GET("/", eventApp.Get)
event.GET("/:id", eventApp.Get)
event.PUT("/:id", eventApp.Put)
event.PUT("/:id/:status", eventApp.Put)
event.DELETE("/:id", eventApp.Delete)
goods := g.Group("/goods")
goodsApp := admin.Goods{}
goods.POST("/", goodsApp.Post)
goods.GET("/", goodsApp.Get)
goods.GET("/:id", goodsApp.Get)
goods.PUT("/:id", goodsApp.Put)
goods.DELETE("/:id", goodsApp.Delete)
}
Copy the codeIn addition, we need to start the Admin service and handle the problem of closing the connection when the service exits. Here I have adapted the API. Go directory in the interfaces/ API directory to place the code for interface reuse and program update in proc.go and Listen. go directory in the infrastructure/utils directory for admin reuse. Finally, we implement the admin.go code in interfaces/admin:
var lis net.Listener func Run() error { var err error bind := viper.GetString("admin.bind") lis, err = utils.Listen("tcp", bind) if err ! = nil {return err} g := gin.New() // update the program, Send a signal to the old version go utils.updateProc ("admin") // Initialize the route initRouters(g) // Run the service return g.runlistener (lis)} func Exit() { Lis.close () // TODO: wait for the request to be processed // time.sleep (10 * time.second)}Copy the codeNext, we can parse the command in CMD /admin.go and start admin. The code is as follows:
var adminCmd = &cobra.Command{ Use: "admin", Short: "Seckill admin server.", Long: `Seckill admin server.`, Run: func(cmd *cobra.Command, args []string) { onExit := make(chan error) go func() { if err := admin.Run(); err ! = nil { logrus.Error(err) onExit <- err } close(onExit) }() onSignal := make(chan os.Signal) signal.Notify(onSignal, syscall.SIGINT, syscall.SIGTERM) select { case sig := <-onSignal: logrus.Info("exit by signal ", sig) admin.Exit() case err := <-onExit: logrus.Info("exit by error ", err) } }, } func init() { rootCmd.AddCommand(adminCmd) }Copy the codeAt this point, the SECkill API is designed and the most basic framework is implemented. As you can see, I’m not working on a feature all at once, but I’m working from the outside in and verifying it all the time. This process is called progressive development and is a common method in software development.