Preface:

Morning found a fairy blog! Write things very deep, easy to understand, reprint it ~

Reprinted from: blog.csdn.net/qq_41453285…

Thank you ~

1. Description of TCP connections

  • (1) when using TCP protocol between the client and the server will set up a virtual channel and virtual channel is refers to the connection, and suggest the connection requires three times handshake, destroy this link need four waves, visible, we set up this connection has a cost, the cost is the cost efficiency, simple point said is time into this, if you want to send a piece of data, You have to shake hands 3 times (for 3 packets) before you can send the data, and after sending, you need to wave hands 4 times (for 4 packets) to disconnect the connection
  • * * (2) the cost of CPU resources, * * three-way handshake and four wave, and sending data from network card sent and received, and the rest of the equipment, such as firewall, router, etc., stand in terms of the operating system kernel, if we are a high concurrent systems, if a large number of data packets have experienced such a process, That’s CPU intensive.
  • **③ Each socket is required to use the system cache, ** for example, the system provides some interfaces to set the socket cache, such as:
    • /proc/sys/net/ipv4/tcp_rmem
    • /proc/sys/net/ipv4/tcp_wmem
    • /proc/sys/net/ipv4/tcp_mem

  • Due to the reliable transmission of TCP, a large number of applications use TCP as communication protocol, but each application uses TCP differently due to product functions, such as instant chat system (wechat, Dingding, Tantan).

TCP long connection, TCP short connection

TCP short connection

  • ** Concept: ** As shown in the figure below, the client and the server establish a connection to start communication, and then disconnect the TCP connection after one/specified number of communication ends. The next time the client communicates again, the TCP connection is established again
  • ** Advantages: ** does not occupy the memory of the server for a long time, so the number of connections the server can handle is relatively large
  • Disadvantages:
    • ** If the server wants to send data to the client, what can be done? There is no way, or wait until the next time to request data, such as we use polling (30 seconds or longer) pull messages, then the real-time communication between the server and the client will be lost
    • Clients use polling to obtain information in real time, or a large number of clients use short connection to communicate, so a large number of CPU and bandwidth resources are wasted for establishing and releasing connections, there is a waste of resources, or even cannot establish connections. Such as the classic HTTP long polling (wechat web client)

Long TCP connection

  • As shown in the figure below, TCP connects to the server and remains connected until the service is no longer needed
  • Advantages:
    • Fast data transfer
    • The server can actively transfer data to the client in the first time
  • Disadvantages:
    • Because the client and server are always connected in this way, the number of clients in a highly concurrent distributed cluster system will increase and occupy a lot of system resources
    • TCP itself is a kind of stateful data, in high concurrency distributed system will cause the background design is difficult to do

3. TCP keepalive mechanism

  • For details about TCP preservation, see blog.csdn.net/qq_41453285…
  • TCP implements a keepalive mechanism designed to:
    • Do clients and servers need to know when to terminate processes or disconnect from each other
    • In other cases, no data is exchanged between application processes, but a minimal data flow needs to be maintained through connections, which can be very resource-intensive. A keepalive mechanism can be used to detect the other party and close the connection when appropriate
  • The main working principle of ** is: ** the probe will design a timer on its own end, and when the timer is triggered, it will send a probe message to the other side. If the peer sends back an ACK, it indicates that the peer is still alive. If no ACK is sent within the specified time, the TCP connection is disconnected

Kernel parameters for Linux

  • **tcp_keepalive_time: ** Indicates the idle time of the link before sending probe packets. The default value is 7200
  • ** tcp_keepalive_INTvL: ** Indicates the interval between sending probe packets. The default value is 75
  • ** TCP_keepalive_probes: ** indicates the number of probes. The default value is 9

Posix socket options

  • The three kernel parameters described above have corresponding socket options available in Linux programming

  • For setsockopt() see: blog.csdn.net/qq_41453285…

    int keepalive_time = 30; The setsockopt (IPPROTO_TCP incomingsock, TCP_KEEPIDLE, (void *) (& keepalive_time), (socklen_t) sizeof (keepalive_time)); int keepalive_intvl = 3; setsockopt(incomingsock, IPPROTO_TCP,TCP_KEEPINTVL,(void*)(&keepalive_intvl),(socklen_t)sizeof(keepalive_intvl)); int keepalive_probes= 3; setsockopt(incomingsock, IPPROTO_TCP,TCP_KEEPCNT,(void*)(&keepalive_probes),(socklen_t)sizeof(keepalive_probes));

Four, TCP long connection design

Why long connections

  • ** The server must actively send messages to the client: ** If no connection exists, the server will never actively find the client and will not be able to send messages to the client in time
  • ** The client and server communicate frequently: ** If the connection is short, the connection needs to be established each time to send a message. If the concurrency is high, there may be a large number of TIME_WAIT state sockets, which means that the connection cannot be established later
  • ** Business needs, such as when the client goes offline, the server needs to do something: ** such as clearing his cache or other resources or other business meaning, such as QQ avatar display offline

This section describes the problems to be considered when designing TCP long connections

  • ** The default TCP keep-alive timeout is too long: ** The default is 7200 seconds, or 2 hours, which can be changed of course
  • ** Socket proxy disables TCP keep-alive: ** All proxy applications can only forward TCP application data, but cannot forward TCP internal packets
  • ** Mobile networks need to keep signaling alive: ** For smart terminals such as mobile phones, which use mobile networks to access the Internet, operators will try to close sockets that do not send data for more than 60 seconds or 45 seconds in order to save channel resources
  • The following uses the heartbeat detection mechanism to solve these problems

Five, heartbeat detection

  • Heartbeat detection is a mechanism implemented by the user at the application layer. It is used to imitate the KEEPalive mechanism of TCP. The user sends a heartbeat detection packet to the peer within a specified period of time. If no reply is received within the specified time, then the appropriate action is taken (such as closing the TCP connection).

Implementation in Netyy

Coding implementation and main principles

  • Github source link:Github.com/dongyusheng…
  • Code:
    • The code uses a red-black tree to manage timers
    • The whole framework is implemented by Reactor model
    • There are corresponding clients and servers

  • The general idea is as follows:
    • We manage all event nodes with a red-black tree, where key is the timeout of the event socket and value is the socket
    • Wpoll_wait () is called to handle all events, and its last parameter is a timeout, which must be set to the value of the node with the shortest timeout in the entire red-black tree
    • When epoll_wait() returns, the time is compared to the node’s time, and if it timed out, it is processed accordingly

Code parsing

  • The following describes how the server performs heartbeat detection on the client from the first perspective
  • The ngx_reacTOR_loop () function of ngx_reactor. C:
    • The Reactor’s main center handler, ngx_reactor_loop(), gets the current time before poll_wait() is called
    • Epoll_wait () is then performed, whose last argument is the value of the node with the shortest timeout in the red-black tree
    • After epoll_wait() returns, update the current time (global variable ngx_current_msec)
    • Finally, get the current time again and subtract it from the previous old “current time” to get the difference. If the timeout expires, the ngx_EVENT_expire_timers () function is called

  • The ngx_event_expire_timers() function of ngx_reactor.c:
    • This function also is a sign of what the packaging events on a NGX_EVENT_TIMEOUT |, the event setting timeout
    • The event’s callback function is then called

  • In the above callback, we bind it to the client_handler() function in ngx_server.c:
    • This function determines whether the event’s what flag has an NGX_EVENT_TIMEOUT flag and calls the timeout_cb() handler if it does
    • The timeout_cb() function or determines whether delta has exceeded KEEPALIVE_INTVAL_MSEC and has timed out three times (KEEPALIVE_INTVAL_PROBES), if so, close() is called to close the client socket

  • The macros defined above are shown below