[TOC]

This is the third day of my participation in Gwen Challenge

Explore the net/ HTTP code flow

What is NET/HTTP?

GO is one of the standard library, used for the development of Web applications, using this library, can make the development become faster and easier, and easy to use.

So here’s the question

The use of library, really convenient, no brain adjustment interface, put together can run on the line, tube his efficiency performance, out of the problem, delete library run on the line…

Is this really the case? As a developer, we must find a way to understand the things that are not clear, to understand the principle of using tools, but also need to clearly know the operation principle of their own product development, just as the so-called

To know what it is, but not why, and to imitate it, but the heart cannot speak, nor the mouth declare, nor the pen write.Copy the code

For those of us who want to explore technology and be in awe of code, today let’s take a look at the net/ HTTP code flow

To use a framework/library, it is necessary to accept its own set of conventions and patterns, and we must understand and become familiar with how these conventions and patterns are used, otherwise we will be in the situation of not knowing when we are using them wrong.

In GOLANG, the components of NET/HTTP are client and server

Some of the structures and functions in the library support either the client or the server, while others support both.

Only clients are supported

The Client, the response

Only the server is supported

ServerMux, Server, ResponseWriter, Handler and HandlerFunc

Both client side and server side are supported

Header, Request, Cookie

Net/HTTP building server is also very simple, the general framework is as follows:

The client requests the server, which uses the NET/HTTP package, which has the multiplexer, and the corresponding interface of the multiplexer. Multiple processors in the server process different requests, and finally the data that needs to be dropped is stored in the library

The Great Wall first step, we set off

Start writing a simple Request

package main

import (
   "fmt"
   "net/http"
)

func main(a) {

   http.HandleFunc("/Hi".func(w http.ResponseWriter, r *http.Request) {
       // 
 are HTML tags
      w.Write([]byte("<h1>Hi xiaomotong</h1>"))})// ListenAndServe Do not write IP The default server address is 127.0.0.1
   if err := http.ListenAndServe(": 8888".nil); err ! =nil {
      fmt.Println("http server error:", err)
   }
    
}
Copy the code

Run the server code and type 127.0.0.1:8888/Hi or localhost:8888/Hi in your browser to see the effect

Creating a Go write server is as simple as calling ListenAndServe and passing in the network address, port, and handler to handle the request.

Note:

If the network address parameter is an empty string, the server defaults to using port 80 for network connection
If the processor parameter isnil, the server will use the default multiplexerDefaultServeMux.

But how is HTTP actually set up? A fierce operation such as tiger, a detail 250

HTTP setup process

The HTTP setup process is universal because it is a standard protocol. When writing C/C++, these processes are basically written by themselves, but when writing GO, the standard library is already wrapped, hence the situation where a single function can write a Web server

Processes involved on the server side

Socket Sets up a socket
Bind Bind address and port
Listen Sets the maximum number of listeners
Accept starts blocking the connection waiting for the client
Read Reads data
Write Write back data
Close close

Processes involved in the client

Socket Sets up a socket
Connect Connects to the server
The write write data
Read Reads data

So how does the data move across the hierarchy?

The familiar 7-tier OSI model is used, but in practice we use the TCP/IP 5-tier model

The protocols that may be used in the TCP/IP five-tier model are listed in general

Application layer:

HTTP protocol, SMTP, SNMP, FTP, Telnet, SIP, SSH, NFS, RTSP

The transport layer

Common protocols are TCP, UDP, SCTP, SPX, ATP, etc

UDP is unreliable, and SCTP has its own special application scenarios. Therefore, HTTP is generally transmitted by TCP

However, enterprise applications will transform UDP into reliable transport, in fact, on standard UDP encapsulation custom header, analog TCP reliable transport, three handshake, four wave is happening here

The network layer

IP protocol, ICMP, IGMP, IPX, BGP, OSPF, RIP, IGRP, EIGRP, ARP, RARP, etc

Data link layer

Ethernet, PPP, WiFi, 802.11 and so on

The physical layer

SO2110, IEEE802 and so on

Knowing the general flow of HTTP, let’s take a look at how the net/ HTTP library implements the whole flow, starting with the establishment of sockets

Net/HTTP socket is established

Remember that little request case at the top? This is where we can start

package main

import (
   "fmt"
   "net/http"
)

func main(a) {

   http.HandleFunc("/Hi".func(w http.ResponseWriter, r *http.Request) {
      w.Write([]byte("<h1>Hi xiaomotong</h1> "))})if err := http.ListenAndServe(": 8888".nil); err ! =nil {
      fmt.Println("http server error:", err)
   }
}
Copy the code

http.HandleFunc(“/Hi”, func(w http.ResponseWriter, r *http.Request) { w.Write([]byte(“

Hi xiaomotong

“)})

The HandleFunc section is the registered route, and the handler for this route is placed in DefaultServeMux by default

// HandleFunc registers the handler function for the given pattern
// in the DefaultServeMux.
// The documentation for ServeMux explains how patterns are matched.
func HandleFunc(pattern string, handler func(ResponseWriter, *Request)) {
   DefaultServeMux.HandleFunc(pattern, handler)
}
Copy the code

HandleFunc is actually HandleFunc that calls the ServeMux service

// HandleFunc registers the handler function for the given pattern.
func (mux *ServeMux) HandleFunc(pattern string, handler func(ResponseWriter, *Request)) {
   if handler == nil {
      panic("http: nil handler")
   }
   mux.Handle(pattern, HandlerFunc(handler))
}
Copy the code

The HandleFunc of the ServeMux service calls the Handle implementation of its own service

// Handle registers the handler for the given pattern.
// If a handler already exists for pattern, Handle panics.
func (mux *ServeMux) Handle(pattern string, handler Handler) {
   mux.mu.Lock()
   defer mux.mu.Unlock()

   if pattern == "" {
      panic("http: invalid pattern")}if handler == nil {
      panic("http: nil handler")}if _, exist := mux.m[pattern]; exist {
      panic("http: multiple registrations for " + pattern)
   }

   if mux.m == nil {
      mux.m = make(map[string]muxEntry)
   }
   e := muxEntry{h: handler, pattern: pattern}
   mux.m[pattern] = e
   if pattern[len(pattern)- 1] = ='/' {
      mux.es = appendSorted(mux.es, e)
   }

   if pattern[0] != '/' {
      mux.hosts = true}}Copy the code

Seeing the actual registered routing implementation is still relatively simple, we will not go deep into the network, if interested can continue here to follow the specific data structure

The current route registration process is as follows:

http.HandleFunc ->
(mux *ServeMux) HandleFunc ->
(mux *ServeMux) Handle

Net/HTTP listening port + response request

Let’s take a look at the address and port code in the request example

if err := http.ListenAndServe(“:8888”, nil); err ! = nil { fmt.Println(“http server error:”, err) }

After the above three function flows, we already know how to register the route, thenListenAndServeHow does this function respond after the handler has processed the data? Come on, let’s move on.

ListenAndServe listens on TCP network address ADDR, then calls handler to handle incoming connection requests, the received connection is configured to enable TCP keep-alive, this parameter is usually nil, in this case DefaultServeMux is used, as mentioned above, Here again

// ListenAndServe listens on the TCP network address addr and then calls
// Serve with handler to handle requests on incoming connections.
// Accepted connections are configured to enable TCP keep-alives.
//
// The handler is typically nil, in which case the DefaultServeMux is used.
//
// ListenAndServe always returns a non-nil error.
func ListenAndServe(addr string, handler Handler) error {
   server := &Server{Addr: addr, Handler: handler}
   return server.ListenAndServe() ListenAndServe (SRV *Server) ListenAndServe (SRV *Server
}
Copy the code

// ListenAndServe listens on the TCP network address srv.Addr and then
// calls Serve to handle requests on incoming connections.
// Accepted connections are configured to enable TCP keep-alives.
//
// If srv.Addr is blank, ":http" is used.
//
// ListenAndServe always returns a non-nil error. After Shutdown or Close,
// the returned error is ErrServerClosed.
func (srv *Server) ListenAndServe(a) error {
   if srv.shuttingDown() {
      return ErrServerClosed
   }
   addr := srv.Addr
   if addr == "" {
      addr = ":http"
   }
   ln, err := net.Listen("tcp", addr) // Net.listen is used to Listen to addresses and ports
   iferr ! =nil {
      return err
   }
   return srv.Serve(ln)
}
Copy the code

func Listen(network, address string) (Listener, error) {
   var lc ListenConfig
   return lc.Listen(context.Background(), network, address)
}
Copy the code

func (srv *Server) Serve(l net.Listener) error {
   iffn := testHookServerServe; fn ! =nil {
      fn(srv, l) // Call hook with unwrapped Listener // The position where the callback function is called
   }

 / /... 15 lines of code omitted here...
    
   var tempDelay time.Duration // How long to sleep on accept failure
	
   ctx := context.WithValue(baseCtx, ServerContextKey, srv)
   for {
       // Start Accept blocks the connection to the listening client
      rw, err := l.Accept()
      iferr ! =nil {
         select {
         case <-srv.getDoneChan():
            return ErrServerClosed
         default:}if ne, ok := err.(net.Error); ok && ne.Temporary() {
            if tempDelay == 0 {
               tempDelay = 5 * time.Millisecond
            } else {
               tempDelay *= 2
            }
            if max := 1 * time.Second; tempDelay > max {
               tempDelay = max
            }
            srv.logf("http: Accept error: %v; retrying in %v", err, tempDelay)
            time.Sleep(tempDelay)
            continue
         }
         return err
      }
      connCtx := ctx
      ifcc := srv.ConnContext; cc ! =nil {
         connCtx = cc(connCtx, rw)
         if connCtx == nil {
            panic("ConnContext returned nil")
         }
      }
      tempDelay = 0
      c := srv.newConn(rw)
      c.setState(c.rwc, StateNew, runHooks) // before Serve can return
      go c.serve(connCtx)	// Here a coroutine is opened to handle the specific request message}}Copy the code

Here a coroutine is turned on to handle specific request messages by go c.sever (connCtx)

// Serve a new connection.
func (c *conn) serve(ctx context.Context) {
   c.remoteAddr = c.rwc.RemoteAddr().String()
   ctx = context.WithValue(ctx, LocalAddrContextKey, c.rwc.LocalAddr())
   / /... Some code is omitted here
    // HTTP cannot have multiple simultaneous active requests.[*]
    // Until the server replies to this request, it can't read another,
    // so we might as well run the handler in this goroutine.
    // [*] Not strictly true: HTTP pipelining. We could let them all process
    // in parallel even if their responses need to be serialized.
    // But we're not going to implement HTTP pipelining because it
    // was never deployed in the wild and the answer is HTTP/2.
    HTTP cannot have multiple active requests at the same time. [*], until the server responds to this request, it cannot read another
    serverHandler{c.server}.ServeHTTP(w, w.req)  // ServeHTTP is the focus
    w.cancelCtx()
    if c.hijacked() {
        return
    }
    w.finishRequest()
    / /... Some code is omitted here
}
Copy the code

ServeHTTP is important here

// serverHandler delegates to either the server's Handler or
// DefaultServeMux and also handles "OPTIONS *" requests.
type serverHandler struct {
   srv *Server
}
// Process the request
func (sh serverHandler) ServeHTTP(rw ResponseWriter, req *Request) {
   handler := sh.srv.Handler
   if handler == nil {
      handler = DefaultServeMux
   }
   if req.RequestURI == "*" && req.Method == "OPTIONS" {
      handler = globalOptionsHandler{}
   }
   handler.ServeHTTP(rw, req)
}
Copy the code

// ServeHTTP dispatches the request to the handler whose
// pattern most closely matches the request URL.
func (mux *ServeMux) ServeHTTP(w ResponseWriter, r *Request) {
   if r.RequestURI == "*" {
      if r.ProtoAtLeast(1.1) {
         w.Header().Set("Connection"."close")
      }
      w.WriteHeader(StatusBadRequest)
      return
   }
   h, _ := mux.Handler(r)
   h.ServeHTTP(w, r)
}
Copy the code

(sh serverHandler) ServeHTTP call (mux *ServeMux) ServeHTTP, ServeHTTP sends the request to the Handler h, _ := mux.handler (r)

Source code see here, for net/ HTTP standard library for registering routes, listening to server address and port process, roughly clear it

The whole process, NET/HTTP basically provides the whole HTTP flow of services, you can say very fragrant, the whole process basically looks like this

Net. Listen initializes sockets, binds IP addresses to ports, and listens to set the maximum number of listeners
Accept blocks waiting for the client to connect
Go C.sever (CTX) starts a new coroutine to process the current request. At the same time, the main coroutine continues to wait for other clients to connect and perform high-concurrency operations
Mux. Handler Obtains the registered route, obtains the Handler of this route, processes the request from the client, and returns the result to the client

About how to unpack the underlying packet, how to offset bytes, how to deal with ipv4 and ipv6, interested friends can continue to follow the code, welcome more communication

Well, that’s all for now. Next time we’ll share gin’s routing algorithm,

Technology is open, our mentality, should be more open. Embrace change, live in the sun, and strive to move forward.

I am Nezha, welcome to like, see you next time ~

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

Let’s explore the net/ HTTP code flow

Explore the net/ HTTP code flow

What is NET/HTTP?

The Great Wall first step, we set off

Start writing a simple Request

HTTP setup process

Net/HTTP socket is established

Hi xiaomotong

Let’s explore the net/ HTTP code flow

Explore the net/ HTTP code flow

What is NET/HTTP?

The Great Wall first step, we set off

Start writing a simple Request

HTTP setup process

Net/HTTP socket is established

Hi xiaomotong

Related Posts

You don’t even know bin log or redo log

FFmpeg notes – Basic use

Spring Cloud Gateway Loadbalancer