This article is participating in Python Theme Month. See the link for details

background

The following questions will help you better understand the meaning of Python Socket programming and the framework:

  • What is the essential difference between TCP and UDP?

TCP protocol, connection-oriented, reliable, transmission layer communication protocol based on byte stream; UDP is a connectionless, unreliable, datagram – based transport layer protocol. TCP requires three handshakes to establish a connection and four handshakes to disconnect a connection, which increases the security of the transmission process. However, the process of establishing a connection consumes system resources and consumes more time, while UDP transmission does not appear this problem. To sum up, in TCP transmission, the receiver needs to continuously confirm whether the receiver has received the message to establish a connection (the number of confirmation processes is limited, that is, three-way handshake). In UDP transmission, the receiver does not need to confirm whether the receiver has received the message, but only needs to send the message to the receiver.

  • What are the differences and connections between TCP/IP protocol stack, HTTP protocol and Socket?

TCP/IP stack is a series of network protocols, which can be divided into four layers to analyze: application layer, transport layer, network layer, link layer; HTTP protocol (Hypertext Transfer Protocol) is the application layer protocol in this protocol stack. HTTP protocol in simple terms, its role is to standardize the format of data, so that programs can be easily identified, and both the sender and the receiver need to follow the same protocol format for data transmission. (The application layer protocol is similar to HTTP, except that it defines different data formats.) Socket refers to the external operation interface provided by the TCP/IP protocol stack, that is, the interface used by the application layer to communicate with each other through network protocols. Socket can use different network protocols for end-to-end communication.

  • How does the TCP Socket server communicate?

Server side: Establish a connection (socket() creates a socket descriptor, bind() binds a specific listening address (IP +port), listen() listens on the socket, and accept() blocks and waits for the client to connect) Data interaction (the read() function blocks and waits for the Client to send data, while the write() function sends data to the Client.) Establish a connection (socket() creates a socket descriptor, connect() sends a connection request to the specified listening address) data interaction (Wirte () sends server data, read() blocks data waiting to be sent by the server)

  • The connection between sockets and websockets?

Webosocket is a communication protocol, different from HTTP request, the client requests server resources, the server responds to the communication process; Websocket allows the server to actively push messages to the client, and achieves two-way communication between the client and the server. (The specific underlying principles need to be practiced later, so I haven’t contacted them for the time being.)

  • What are the connections and differences between HTTP and WSGI protocols?

HTTP (Hypertext Transfer Protocol) is a protocol at the application layer of the TCP/IP stack. It is a client – server – side transfer rule that regulates the format of data to be transferred. The WSGI protocol is a Python-defined interface rule for Web servers and framework applications to communicate. The two are not closely related, and the Python framework, by definition, handles HTTP requests. (A Python framework for the WSGI protocol can be implemented later to experiment with HTTP requests.)

  • Mainstream Web Frameworks, asynchronous Web frameworks?

Mainstream Web frameworks: Django, Flask Asynchronous Web frameworks: Tornado (built-in asynchronous module), Snaic (Python asyncio), FastAPI (based on Starlette library), AIOHTTP (based on Asyncio)

  • The connection between asyncio and AIoHTTP? (Asynchronous programming)

Asyncio is an asynchronous IO library. Aiohttp is an asynchronous HTTP framework based on Asyncio (supporting client/server).

Code design

Python provides basic socket modules:

  1. The socket module; Provides a standard BSD Sockets API;
  2. Socketserver module: provides the server center class, simplifies the server development;

TCP Socket server

The socket module:

# -*- coding: utf-8 -*-
from socket import socket, AF_INET, SOCK_STREAM

def echo_handler(sock ,address) :
	print("Get Connection from address:", address)

	while True:
		response = sock.recv(8192)
		if not response:
			break
		print(f"Got {response}")
		sock.sendall(response)

def echo_server(address, back_log=5) :
	sock = socket(AF_INET, SOCK_STREAM)
	sock.bind(address)
	sock.listen(back_log)

	while True:
		sock_client, address = sock.accept()
		echo_handler(sock_client, address)

if __name__ == "__main__":
	echo_server(('localhost'.5000))
Copy the code

Code details:

  • Create a Socket based on both IPV4 and TCP. AF_INET indicates IPV4, SOCK_STREAM indicates stream-oriented TCP, bind listening ports, and set the maximum number of waiting connections
  • Create a permanent loop that gets the connection requested by the client, and accept() waits and returns a client connection;
  • After the connection is established, wait for the client data, receive the client data, return the data to the client, and close the connection

Problems: When multiple client requests occur, a single thread will block, so if multiple threads need to process multiple client requests, you can change this;

from threading import Thread

while True:
        client_sock, address = sock.accept()
        thread = Thread(target=echo_handler, args=(client_sock, address))
        thread.start()
Copy the code

This way, a child thread is generated and processed for each client request; (There is a problem, however: when a sudden flood of requests for connections reaches the upper limit of system resources, the program may not be able to handle subsequent requests.)

Socketserver module:

# -*- coding: utf-8 -*-

from socketserver import BaseRequestHandler, StreamRequestHandler
from socketserver import TCPServer, ThreadingTCPServer


class SingleHandler(BaseRequestHandler) :
	def handle(self) :
		print("Got Connections From: %s" % str(self.client_address))
		
		while True:
			msg = self.request.recv(8192)
			print(f"Got msg: {msg}")
			if not msg:
				break
			self.request.send(msg)

class StreamHandler(StreamRequestHandler) :
	def handle(self) :
		print("Got Connection From: %s" % str(self.client_address))
		
		for line in self.rfile:
			print(line)
			self.wfile.write(line)


if __name__ == "__main__":
	# server = TCPServer(("", 5000), SingleHandler)
	# server = TCPServer(("", 5000), StreamHandler)
	server = ThreadingTCPServer(("".5000), StreamHandler)
	server.serve_forever()
Copy the code

Code details:

  • To process multiple clients, initialize an instance of ThreadingTCPServer, which handles client connections and creates a thread for each client to interact with.
  • Set the bound IP address and port, as well as the processing class;
  • You can use BaseRequestHandler directly, the parent of all request handling classes. Subclasses override handle() to handle requests. This module is combined with the service class to handle requests.
  • Use StreamRequestHandler (a stream-like request handler class that provides a standard file interface to simplify communication) to override the Handle method to retrieve request data and return it to the client.
  • When working with a class using StreamRequestHandler, recv() is called multiple times while reading data sent by the client until a newline character is encountered, so the client needs to add a newline character at the end of the data sent

TCP Socket client

The socket module:

# -*- coding: utf-8 -*-
from socket import socket, AF_INET, SOCK_STREAM
import time

def request_handler() :
	start_time = time.time()
	sock_client = socket(AF_INET, SOCK_STREAM)
	sock_client.connect(('localhost'.5000))
	
	book_content = ""
	with open("send_books.txt"."r") as f:
		book_content = f.read()
	
	content_list = book_content.split("\n")
	for content in content_list:
		if content:
			A newline character should be added to the end of each message sent for StreamRequestHandler recognition
			sock_client.send((content+"\n").encode())
			time.sleep(1)
			response = sock_client.recv(8192)
			print(response)

	end_time = time.time()
	print("Total time :", end_time-start_time)

if __name__ == "__main__":
	request_handler()
Copy the code

UDP Socket

The Socket module:

# -*- coding: utf-8

from socket import socket, AF_INET, SOCK_DGRAM


def echo_handler(address) :
	server = socket(AF_INET, SOCK_DGRAM)
	server.bind(address)
	
	while True:
		msg, addr = server.recvfrom(8192)
		if not msg:
			break

		print(f"Got Message From: {addr} \n {msg}")
		server.sendto(msg, addr)

if __name__ == "__main__":
	echo_handler(("".8888))
Copy the code

The code is not explained in detail, and the previous similar, pay attention to the different protocol is done

Client tests:

# -*- coding: utf-8 -*-

from socket import socket, AF_INET, SOCK_DGRAM
import time


def request_handler(addr) :
	client = socket(AF_INET, SOCK_DGRAM)
	
	book_content = ""
	with open("send_books.txt"."r") as f:
		book_content = f.read()

	book_list = book_content.split("\n")
	for content in book_list:
		if content:
			client.sendto(content.encode(), addr)
			response = client.recv(8192)
			print(response)
		time.sleep(1)

if __name__ == "__main__":
	addr = ("localhost".8888)
	request_handler(addr)
Copy the code

Socketserver module:

# -*- coding: utf-8 -*-
from socketserver import BaseRequestHandler, UDPServer

class EchoHandler(BaseRequestHandler) :

	def handle(self) :
		print(f"Got Connections From: {self.client_address}")	

		data, sock = self.request
		print(data)

		if data:
			sock.sendto(data, self.client_address)

if __name__ == "__main__":
	server = UDPServer(("".8888), EchoHandler)
	server.serve_forever()
Copy the code

Without going into the code, ThreadingUDPServer can be used if you need multiple threads to handle concurrent operations

conclusion

This introduction to Python Socket programming is mostly superficial. It only talks about several modules of Python’s actual Socket processing. The underlying knowledge of Socket is not mentioned. Understands the role of sockets in the entire TCP/IP stack. Socket and SocketServer modules can be used to write network programs, but socketServer is a lot easier. You can focus on business logic and don’t have to worry about the details of sockets, including but not limited to multithreading/multi-process, receiving data, sending data, communication process.