An overview of the
Go-restful is a web framework developed in go language to quickly build restful style. K8s the most core component kube-Apiserver to use the framework, the framework of the code is relatively simple, here to do a simple function, and then analyze the relevant source code.
Go restful is based on golang’s official NET/HTTP implementation. Before further study, I suggest that we take a look at my previous article on official HTTP source code analysis
Go-restful defines three important data structures:
- Router: indicates a route, including the URL and callback processing function
- Webservice: indicates a service
- Container: Indicates a server
The relationship among the three is as follows:
- Go restful supports multiple Containers. One Container is like an HTTP server. Different Containers monitor different addresses and ports
- Each Container can contain multiple WebServices, representing a taxonomy of different services
- Each WebService contains multiple routers, which are routed to the corresponding Handler function (Handler Func) based on the URL of the HTTP request.
Hd address
Quick learning
The introduction of package:
go get github.com/emicklei/go-restful/v3
Copy the codeExample hello-world code: loclahost:8080/hello
package main
import (
"github.com/emicklei/go-restful/v3"
"io"
"log"
"net/http"
)
func main(a) {
/ / create a WebService
ws := new(restful.WebService)
// Set the route and callback functions for the WebService
ws.Route(ws.GET("/hello").To(hello))
// Add WebService to the Container generated by default
// The default generated container code is in the init method of web_service_container.go
restful.Add(ws)
// Start the service
log.Fatal(http.ListenAndServe(": 8080".nil))}// The callback function corresponding to the route
func hello(req *restful.Request, resp *restful.Response) {
io.WriteString(resp, "world")}Copy the codeSource code analysis
After creating WebServie and adding it to the default Container, you don’t pass the Container to anyone. You just start the service listener and it automatically identifies the Container.
To uncover the answer, let’s analyze the source code. Before I do that, I recommend reading my previous article on official HTTP source code analysis, because Go restful implements functionality based on official HTTP packages
The figure below is the core logic diagram of the source code.
Hd address
Core data structure
Route
As mentioned earlier, Route is one of the three concepts of GO-restful. The internal data structure is Route. Look at the source code first.
Source: github.com/emicklei/go-restful/router.go
type Route struct {
Method string
Produces []string
Consumes []string
// Requested path: root path + described path
Path string
// handler handles the function
Function RouteFunction
/ / the interceptor
Filters []FilterFunction
If []RouteSelectionConditionFunction
// cached values for dispatching
relativePath string
pathParts []string
pathExpr *pathExpression // cached compilation of relativePath as RegExp
// documentation
Doc string
Notes string
Operation string
ParameterDocs []*Parameter
ResponseErrors map[int]ResponseError
DefaultResponse *ResponseError
ReadSample, WriteSample interface{} // structs that model an example request or response payload
// Extra information used to store custom information about the route.
Metadata map[string]interface{}
// marks a route as deprecated
Deprecated bool
//Overrides the container.contentEncodingEnabled
contentEncodingEnabled *bool
}
Copy the codeRouteBuilder
The RouteBuilder is used to construct the Route information, using the builder pattern as known by name
Source: github.com/emicklei/go-restful/router.go
// Most attributes are the same as Route
type RouteBuilder struct {
rootPath string
currentPath string
produces []string
consumes []string
httpMethod string // required
function RouteFunction // required
filters []FilterFunction
conditions []RouteSelectionConditionFunction
typeNameHandleFunc TypeNameHandleFunction // required. }Copy the codeWebservice
A WebService has a set of routes that have a common rootPath and logically group together routing requests that have the same prefix
Source: github.com/emicklei/go-restful/web_service.go
type WebService struct {
// Routes in Webservice share a rootPath
rootPath string
pathExpr *pathExpression // cached compilation of rootPath as RegExp
routes []Route
produces []string
consumes []string
pathParameters []*Parameter
filters []FilterFunction
documentation string
apiVersion string
typeNameHandleFunc TypeNameHandleFunction
dynamicRoutes bool
// Protect routes against concurrency problems in multithreaded write operations
routesLock sync.RWMutex
}
Copy the codeContainer
A Container contains multiple services. Different Containers listen on different IP addresses or ports to provide independent services.
Source: github.com/emicklei/go-restful/container.go
type Container struct {
webServicesLock sync.RWMutex
// There are multiple WebServices inside the Container
webServices []*WebService
ServeMux *http.ServeMux
isRegisteredOnRoot bool
containerFilters []FilterFunction
doNotRecover bool // default is true
recoverHandleFunc RecoverHandleFunction
serviceErrorHandleFunc ServiceErrorHandleFunction
router RouteSelector // default is a CurlyRouter (RouterJSR311 is a slower alternative)
contentEncodingEnabled bool // default is false
}
Copy the codeComb the core code process
Start with the code in the previous demo and analyze the call flow.
The overall process includes:
- Create a WebService object
- Add routing addresses and handlers for the WebService object
- Add WebService to Container (no Containerr declared, default Container used)
- Start the service listening port and wait for the service request
func main(a) {
ws := new(restful.WebService)
ws.Route(ws.GET("/hello").To(hello))
restful.Add(ws)
log.Fatal(http.ListenAndServe(": 8080".nil))}Copy the codeWebService adds routes
Ws.route (ws.get (“/hello”).to (hello))
Construct the RouteBuilder object
// The Get method creates a RouteBuilder to construct the Route object
func (w *WebService) GET(subPath string) *RouteBuilder {
// Typical builder pattern usage
return new(RouteBuilder).typeNameHandler(w.typeNameHandleFunc).servicePath(w.rootPath).Method("GET").Path(subPath)
}
// Constructor mode: Assign values to attributes
// The other methods are similar and will not be expanded
func (b *RouteBuilder) typeNameHandler(handler TypeNameHandleFunction) *RouteBuilder {
b.typeNameHandleFunc = handler
return b
}
// After the Get method, the attribute is not completely constructed, and the handler function is assigned with a separate To method
func (b *RouteBuilder) To(function RouteFunction) *RouteBuilder {
b.function = function
return b
}
Copy the codeGenerate a Route object from RouteBuilder
func (w *WebService) Route(builder *RouteBuilder) *WebService {
w.routesLock.Lock()
defer w.routesLock.Unlock()
// Fill in the default values
builder.copyDefaults(w.produces, w.consumes)
// Call the Build method of RouteBuilder to construct a Route
// Add Route to the routes list
w.routes = append(w.routes, builder.Build())
return w
}
The Build method returns a Route object
func (b *RouteBuilder) Build(a) Route{... route := Route{ ... } route.postBuild()return route
}
Copy the codeAdd WebService to Container
Main analysis restful.Add(WS). Special attention should be paid to:
- Pass the HTTP DefaultServeMux to the ServeMux of DefaultServeMux
- Call the Servemux.handlefunc function in Golang’s official HTTP package to process the request
- The processing function is c.dispatch and dispatch, and then routes are distributed internally
Source: github.com/emicklei/go-restful/web_service_container.go
// Define the global variable as the default Container
var DefaultContainer *Container
// init is automatically triggered when another package is imported. That is, when the Go-restful framework is referenced, there is a Container by default
func init(a) {
DefaultContainer = NewContainer()
// The default route object DefaultServeMux under the standard HTTP package in Golang is assigned to the ServeMux of Container
// Pay special attention here, because the logic of this place can answer the previous question. Go -restful and HTTP libraries, using this assignment to establish an association.
DefaultContainer.ServeMux = http.DefaultServeMux
}
// Generate the default container
func NewContainer(a) *Container {
return &Container{
webServices: []*WebService{},
ServeMux: http.NewServeMux(),
isRegisteredOnRoot: false,
containerFilters: []FilterFunction{},
doNotRecover: true,
recoverHandleFunc: logStackOnRecover,
serviceErrorHandleFunc: writeServiceError,
// The default route selector uses the CurlyRouter
router: CurlyRouter{},
contentEncodingEnabled: false}}// Add WebService to default Container
func Add(service *WebService) {
DefaultContainer.Add(service)
}
// Add
func (c *Container) Add(service *WebService) *Container{...// if rootPath was not set then lazy initialize it
if len(service.rootPath) == 0 {
service.Path("/")}// Check whether there is a duplicate RootPath. Different WebServices cannot have the same RootPath
for _, each := range c.webServices {
if each.RootPath() == service.RootPath() {
log.Printf("WebService with duplicate root path detected:['%v']", each)
os.Exit(1)}}if! c.isRegisteredOnRoot {// Core logic: Add handler handler functions for servcie
// Here we pass in c.servemux as the argument, which is the previously mentioned http.defaultServemux
c.isRegisteredOnRoot = c.addHandler(service, c.ServeMux)
}
// Add webServices to the WebService list of the Container
c.webServices = append(c.webServices, service)
return c
}
// addHandler
func (c *Container) addHandler(service *WebService, serveMux *http.ServeMux) bool {
pattern := fixedPrefixPath(service.RootPath())
...
// The key function here: servemux.handlefunc, is a function that implements routing in the Golang standard package
// The route processing function is assigned to the c.dispatch function. It can be seen that this function is the core of the entire Go-restful framework
if! alreadyMapped { serveMux.HandleFunc(pattern, c.dispatch)if! strings.HasSuffix(pattern,"/") {
serveMux.HandleFunc(pattern+"/", c.dispatch)
}
}
return false
}
Copy the codeRoute dispatch function
How to implement hierarchical distribution by Container -> WebService -> Handler? The gO restful framework uses servemux.handlefunc (pattern, C. Dispatch) functions to connect the official HTTP extension mechanism provided by Golang on one hand, and to distribute routes through a Dispatch on the other. So you don’t have to write a lot of handlers on your own.
The core of this function is c.outer.SelectRoute, which finds the appropriate WebService and route based on the request
Source: github.com/emicklei/go-restful/container.go
func (c *Container) dispatch(httpWriter http.ResponseWriter, httpRequest *http.Request){...// Find the most appropriate webService and route according to the request
// This method is described separately later
func(a){... webService, route, err = c.router.SelectRoute( c.webServices, httpRequest) }() ...iferr ! =nil {
// Construct a filter
chain := FilterChain{Filters: c.containerFilters, Target: func(req *Request, resp *Response) {
switch err.(type) {
case ServiceError:
ser := err.(ServiceError)
c.serviceErrorHandleFunc(ser, req, resp)
}
// TODO
}}
// Run the Container filter
chain.ProcessFilter(NewRequest(httpRequest), NewResponse(writer))
return
}
// Try to convert the Router object to the PathProcessor object
// We are using the default Container. The default Container is CurlyRouter.
// One of SelectRoute's implementation classes, RouterJSR311, also implements the PathProcessor. So if you're using RouterJSR311, the interface conversion is the only way to get the value
// The default CurlyRouter does not implement the PathProcessor interface, so the conversion is null and goes to the next if statement
pathProcessor, routerProcessesPath := c.router.(PathProcessor)
if! routerProcessesPath {// Use the default path handler
pathProcessor = defaultPathProcessor{}
}
// Extract parameters from the REQUEST URL request
pathParams := pathProcessor.ExtractParameters(route, webService, httpRequest.URL.Path)
wrappedRequest, wrappedResponse := route.wrapRequestResponse(writer, httpRequest, pathParams)
// Add all filters to the filter chain if there are any
if size := len(c.containerFilters) + len(webService.filters) + len(route.Filters); size > 0 {
// compose filter chain
allFilters := make([]FilterFunction, 0, size)
allFilters = append(allFilters, c.containerFilters...)
allFilters = append(allFilters, webService.filters...)
allFilters = append(allFilters, route.Filters...)
chain := FilterChain{Filters: allFilters, Target: route.Function}
chain.ProcessFilter(wrappedRequest, wrappedResponse)
} else {
// no filters, handle request by route
// The request is processed directly through route without filter
route.Function(wrappedRequest, wrappedResponse)
}
}
Copy the coderouting
The function of c.outer.SelectRoute, mentioned in the previous Dispatch, is to select the appropriate WebService and route, which is described here.
The Router property in the Container is a RouteSelector interface
type RouteSelector interface {
SelectRoute finds a route based on the entered HTTP request and the list of WebServices and returns it
SelectRoute(
webServices []*WebService,
httpRequest *http.Request) (selectedService *WebService, selected *Route, err error)
}
Copy the codeThe Go-restful framework has two implementation classes:
- CurlyRouter
- RouterJSR311
The previous analysis code knows that CurlyRouter is the default implementation, so here we will focus on the SelectRoute function of CurlyRouter
// Select the routing function
func (c CurlyRouter) SelectRoute( webServices []*WebService, httpRequest *http.Request) (selectedService *WebService, selected *Route, err error) {
// Parse the URL into a token list based on '/'
requestTokens := tokenizePath(httpRequest.URL.Path)
// Match the tokens list with the Routing table of the WebService to return the most appropriate WebService
detectedService := c.detectWebService(requestTokens, webServices)
...
// Return the matching set of routes in the WebService
candidateRoutes := c.selectRoutes(detectedService, requestTokens)
...
// Find the best route from the list above
selectedRoute, err := c.detectRoute(candidateRoutes, httpRequest)
if selectedRoute == nil {
return detectedService, nil, err
}
return detectedService, selectedRoute, nil
}
/ / select webservice
func (c CurlyRouter) detectWebService(requestTokens []string, webServices []*WebService) *WebService {
var best *WebService
score := - 1
for _, each := range webServices {
// Calculates the webService score
matches, eachScore := c.computeWebserviceScore(requestTokens, each.pathExpr.tokens)
// Returns the webService with the highest score
if matches && (eachScore > score) {
best = each
score = eachScore
}
}
// Return the webService with the highest score
return best
}
// Calculate the WebService score
func (c CurlyRouter) computeWebserviceScore(requestTokens []string, tokens []string) (bool.int) {
if len(tokens) > len(requestTokens) {
return false.0
}
score := 0
for i := 0; i < len(tokens); i++ {
each := requestTokens[i]
other := tokens[i]
if len(each) == 0 && len(other) == 0 {
score++
continue
}
if len(other) > 0 && strings.HasPrefix(other, "{") {
// no empty match
if len(each) == 0 {
return false, score
}
score += 1
} else {
// not a parameter
ifeach ! = other {return false, score
}
score += (len(tokens) - i) * 10 //fuzzy}}return true, score
}
// Primary: matches path and returns a batch of routes as alternatives
func (c CurlyRouter) selectRoutes(ws *WebService, requestTokens []string) sortableCurlyRoutes {
// Store the selected Route in sortableCurlyRoutes
candidates := make(sortableCurlyRoutes, 0.8)
// Run the webService through all the routes
for _, each := range ws.routes {
// paramCount: matches the regex
// staticCount: A complete match
matches, paramCount, staticCount := c.matchesRouteByPathTokens(each.pathParts, requestTokens, each.hasCustomVerb)
// If it matches, add it to the alternate list
if matches {
candidates.add(curlyRoute{each, paramCount, staticCount}) // TODO make sure Routes() return pointers?}}// Sort alternate routes
sort.Sort(candidates)
return candidates
}
// Secondary filter: matches attributes and other information. Returns the most appropriate Route
func (c CurlyRouter) detectRoute(candidateRoutes sortableCurlyRoutes, httpRequest *http.Request) (*Route, error) {
// tracing is done inside detectRoute
return jsr311Router.detectRoute(candidateRoutes.routes(), httpRequest)
}
// If multiple attributes match: method, content-type, accept
func (r RouterJSR311) detectRoute(routes []Route, httpRequest *http.Request) (*Route, error) {
candidates := make([]*Route, 0.8)
for i, each := range routes {
ok := true
for _, fn := range each.If {
if! fn(httpRequest) { ok =false
break}}if ok {
candidates = append(candidates, &routes[i])
}
}
if len(candidates) == 0 {
if trace {
traceLogger.Printf("no Route found (from %d) that passes conditional checks".len(routes))
}
return nil, NewError(http.StatusNotFound, "404: Not Found")}// Check whether the HTTP method matches
previous := candidates
candidates = candidates[:0]
for _, each := range previous {
if httpRequest.Method == each.Method {
candidates = append(candidates, each)
}
}
if len(candidates) == 0 {
if trace {
traceLogger.Printf("no Route found (in %d routes) that matches HTTP method %s\n".len(previous), httpRequest.Method)
}
allowed := []string{}
allowedLoop:
for _, candidate := range previous {
for _, method := range allowed {
if method == candidate.Method {
continue allowedLoop
}
}
allowed = append(allowed, candidate.Method)
}
header := http.Header{"Allow": []string{strings.Join(allowed, ",")}}
return nil, NewErrorWithHeader(http.StatusMethodNotAllowed, "405: Method Not Allowed", header)
}
// Check whether the content-type matches
contentType := httpRequest.Header.Get(HEADER_ContentType)
previous = candidates
candidates = candidates[:0]
for _, each := range previous {
if each.matchesContentType(contentType) {
candidates = append(candidates, each)
}
}
if len(candidates) == 0 {
if trace {
traceLogger.Printf("no Route found (from %d) that matches HTTP Content-Type: %s\n".len(previous), contentType)
}
if httpRequest.ContentLength > 0 {
return nil, NewError(http.StatusUnsupportedMediaType, "415: Unsupported Media Type")}}// Determine whether accept matches
previous = candidates
candidates = candidates[:0]
accept := httpRequest.Header.Get(HEADER_Accept)
if len(accept) == 0 {
accept = "* / *"
}
for _, each := range previous {
if each.matchesAccept(accept) {
candidates = append(candidates, each)
}
}
if len(candidates) == 0 {
if trace {
traceLogger.Printf("no Route found (from %d) that matches HTTP Accept: %s\n".len(previous), accept)
}
available := []string{}
for _, candidate := range previous {
available = append(available, candidate.Produces...)
}
return nil, NewError(
http.StatusNotAcceptable,
fmt.Sprintf("406: Not Acceptable\n\nAvailable representations: %s", strings.Join(available, ",")))}// If there are multiple matches, return the first one
return candidates[0].nil
}
Copy the codeStart the service
DefaultServeMux, the golang standard library HTTP route object for gO restful direct operation, so this step only needs to call the HTTP standard service start, no additional processing is required. Namely HTTP. ListenAndServe (” : “8080, nil)
conclusion
Go-restful is not a hot Golang Web framework, but K8S used it, this article through the source code analysis of the internal implementation of go-restful do a simple analysis. From the point of view of the analysis process, it is indeed a compact framework. Internal deeper function we did not continue to study, as long as to achieve the purpose of understanding k8S Kube-Apiserver component source code on the line.
The internal core implementation is as long as:
- Add the handler dispatch via DefaultServeMux, the default routing object of the HTTP package
- All route distribution functions are transferred to Dispatch
- Dispatch internally calls the SelectRoute method of CurlyRouter, RouteSelector’s default implementation class, to select the appropriate Route
- The handler method registered with Route is called to process the request