Hi, I’m Alex, and today I’m sharing the first in the computer Networking hardcore series: Understanding Computer Network Communication

This article is mainly on the computer network some of the core ideas of understanding, belonging to internal work heart method, beginners suitable for entry, non-beginners can learn its design ideas, in short, hope to help you improve the understanding of the network;

A simple guide to Linux Networking core

This series from the understanding of network communication to cloud computing network and other network applications of different roles (scenarios), in-depth and simple to discuss the core ideas of network design, network TCP/IP protocol stack design, network programming core technology, network performance optimization, cloud network core technology, network problems troubleshooting, etc. The computer network is a programmer or IT engineer must master the skills, the purpose of this series is to let different readers can deepen the understanding of the network, lay a solid network foundation, no matter in the interview, or the network problems encountered in the work, can be handy.

Understanding of computer network communication

The nature of computer network communication

In 1948 Shannon wrote in The Mathematical Theory of Communication:

“The fundamental problem of communication is the exact or approximate reproduction of selected information at one point at another.”

The essence of network communication is the dissemination of information:

Here the communication medium between A and B can be understood at many different levels

  • Can be wired (twisted pair, cable, optical fiber, etc.)

  • Can be wireless (Bluetooth, WIFI, 2345G, etc.)

  • Can be concrete (physical direct connection)

  • Can be indirectly abstract (across the network)

We can look at the computer network communication model:

This diagram reveals the core elements of computer network communication and the core problems to be solved:

Sender: Producer of data

  • Information coding

  • Data sent

How do I send data?

To send data, you need to use the hardware of the network card. The OS encapsulates the data in accordance with the RFC standard format, and then transmits the data to the network card. The network card converts the data into electrical signals or electromagnetic waves and transmits them out.

Data: Information carrier

  • 01 bit stream

  • Coding format

How do YOU define the data format?

Packets are encapsulated according to standards, and all network devices speak the same “language” throughout the link. In this way, users can communicate freely.

Transmission channel: data transfer station

  • Data is stored

  • The data transport

How to forward data?

Due to sender and receiver:

  1. A sender may send to multiple receivers, or many-to-many scenarios. The sender port is limited, so each cannot communicate independently, and the cost is high and it is not easy to operate.

  2. Long-distance transmission of two devices, it is impossible to directly connect the network cable, cost-effective is too low;

At this point the data needs to be forwarded by intermediate nodes. The transfer node usually transfers from one network port to another network port, which is usually handled by proprietary hardware and can achieve wire speed forwarding (like “direct connection speed”).

Receiver: Consumer of data

  • Data reception

  • Decoding information

How do I receive data?

Also want to receive data, need to use the nic this hardware, the electrical signal/electromagnetic signal into data to OS, OS will be in accordance with the RFC standard format to parse, finally the real data to the application;

Computer network communication principle is relatively “simple” in nature, easy to understand, if let you to design a global network communication system, similar to today’s Internet, how would you design? Would you consider the following questions?

  1. How should the protocol be designed?

  2. What is the packet forwarding mechanism?

  3. How do I guarantee that the system won’t crash?

  4. How to meet the requirements of different types of computer communication?

Internet success is obvious, it has become today’s society is the most important global digital infrastructure, change every aspect of our life, now computer, notebook, TV, ipod, mobile phones can easily get to the Internet, network, information can reach everywhere, what is behind this core technology to support the world’s largest network communication system. In case you’re wondering, let’s take a look at how the Internet solves these problems.

The core idea of computer network communication

Complexity of networks

  • Long distance transmission, such as visiting foreign websites, data across thousands of kilometers, how to ensure data reachable?

  • Will the network crash as the size of the device increases and the amount of data traffic increases?

  • Computer performance is different, how to ensure the fairness of the network?

  • .

How do computer networks solve these problems?

Distributed thinking – connectionless packet switching

Before this, data communication is based on the idea of circuit switching, just like in the traditional telephone network, a private line is established before the call, the delay is small, suitable for real-time telephone communication scenarios, circuit switching embodies a centralized management idea, all communication resources are centrally allocated and managed. But is it suitable for computer network communication?

Technological evolution of data communication

                                   

Circuit Switching (English: Circuit Switching) is a concept relative to message Switching (or packet Switching). Circuit switching requires the establishment of a connection channel between the communication parties. Communication between the two parties can only begin after the connection is successfully established. The communication between the two parties is carried out through the established connection, which is maintained until the end of the communication between the two parties. This connection will remain occupied throughout the duration of a communication activity. The resources (channels, bandwidths, time slots, code words, etc.) assigned to a communication system at the beginning of connection establishment also illustrate the essential characteristics of circuit switching that distinguish it from message switching.

Circuit switching is not suitable for computer network communication despite its high real-time performance:

  • The cost of establishing a proprietary connection is high, and resources need to be allocated to each node of the whole link. The computer network communication scenario is complex, with high frequency and low flow, and it is difficult to predict resources, and resource management will be very complicated.

  • Circuit switching adopts the principle of time division multiplexing, which is completely unable to cope with the traffic burst of modern Internet. At the same time, a large amount of energy will be wasted by idle time slots, and the utilization of link resources is not high. It can be said that the traditional telecommunication network is not scalable.

Message switching improves the utilization of communication lines and does not require a connection, but it is also not suitable for computer network communication:

  • Message switching based on packet data exchange unit, a message with destination address, source address and other information, the entire message to send, a jump, the switching nodes using store-and-forward mode of transmission, cable communication using the transmission technology, because there is no limit to the packet length, and receive from each intermediate node to complete the whole message, when output line is not free, Several complete packets may also be stored for forwarding, requiring a large buffer for each node in the network. In order to reduce the cost and reduce the capacity of node buffer memory, sometimes the message waiting for forwarding is stored on disk, which further increases the transmission delay.

Eventually computer networks adopted packet switching technology:

  • Based on packet switching, packets are divided into smaller data units: packet packet (also known as segment, packet and packet). Packet switching has a shorter delay than packet switching, which is especially suitable for burst data communication between computers.

  • Packet switching uses a distributed idea. Each computer is a “node”, and each node is connected to each other to form a network structure that criss-crosses each other. In this network structure, there is no central concept, and the importance of a single node is greatly reduced. Each node only cares about its neighboring nodes, not the whole link. In this way, it is easier for new nodes to join the network, and the addition of new nodes increases path selection, and enhances the fault tolerance of the network.

  • There is no need to establish a connection, and data packets are independent of network links (each packet carries address information). Links are reusable and resource utilization is high.

  • All forwarding nodes adopt store-and-forward mode. Network nodes are only responsible for exchanging data packets, and the function of business response is pushed to devices at the edge of the network. Data integrity is ensured by the upper layer, which greatly reduces the design of network nodes.

  • Packet switching adopts the principle of statistical reuse, which means that the arrival distribution of packet packets at a processing node conforms to poisson distribution. Combined with the store-and-forward mechanism, the node can cache a certain amount of packets, so as to effectively avoid packet loss caused by delayed processing and ensure the stability and balance of the entire network.

Packet switching (English: Packet switching) is a communication paradigm as opposed to circuit switching, in which packets (also known as messages, or message fragments) are routed individually between nodes without the need to establish a communication path before transmission. Packet switching is a new and important concept in data communication. It is now the most important foundation of Internet communication, data and voice communication in the world. Prior to this, data communication was based on the idea of circuit switching, just as in traditional telephone circuits, a private line was established before a call was made, with the two communicating parties at opposite ends of the circuit. Packet-switching technology emerged in the late 1960s, when the U.S. Advanced Research Projects Agency (ARPA) funded the construction of an experimental network, called ARPANET, for exchanging information between remote computers. One of the main research achievements of ARPANET is to develop a new network protocol, which must be used for conversations on ARPANET networks. The protocol uses a new network information transmission technology, which is packet switching technology.

Computer network communication

How does understanding packet switching help us understand networks?

  • The whole Internet is a large distributed system, TCP/IP protocol is designed from this perspective, congestion control algorithm through self-concession mechanism (non-selfish thought, one for all, all for one), to ensure the fairness and stability of the entire network;

  • At the forwarding level, the network device only cares about the nodes connected to it. The nodes will throw the packets to the next hop device according to the forwarding rules and the longest route matching according to the unified forwarding mechanism. The configuration is configured or learned dynamically by the administrator.

  • The forwarding behavior of network nodes is not centrally controlled and there is no God perspective, but the network can finally realize data forwarding, which reflects some philosophical thinking. The network can connect any unknown node through the relationship around nodes (six degrees theory), which may be the underlying theory for the success of packet switching.

Layered idea – Network protocol layered design

How do you design network protocols?

  • The benefits of layering are reduced coupling. The upper layer does not care about the lower layer implementation, but only the services (interfaces) provided by the lower layer.

  • This allows layer to layer communication to be standardized. Standardization implies layer to layer independence. For example, the transport layer has TCP/UDP/DCCP/STCP, the network layer has IPv4/IPv6, the data link layer has Ethernet, VLAN, WIFI, wireless 3G, 4G, 5G protocols, etc.

  • The network is heterogeneous and complex. Layered networks can integrate heterogeneous networks to provide a unified view to the upper layer. For example, the intermediate IP layer is adapted so that the transport layer and link layer are not entangled and can develop independently.

  • Layered mode has the recursive feature, which allows any encapsulation and reencapsulation of logical meanings, such as overlay network, VPN, and various tunnels, greatly enhancing network scalability.

How does understanding the layered design of computer networks help us understand networks?

1. Each layer only cares about its own logical execution and is not affected by other layers

2. Network problems can be analyzed from top to bottom or from bottom to top

Layered packet encapsulation:

         

Application layer: need to use socket programming interface to send application data;

Transport layer (Layer 4) : Layer 4 uses TCP/UDP and other transmission protocols for data transmission.

Network layer (Layer 3) : Layer 3 mainly implements routing and forwarding.

The routing table mainly stores routing entries. The IP layer adopts the longest matching algorithm to query routes and then forward them:

Link layer (Layer 2) :

Assemble packets according to the link layer protocol (such as Ethernet) corresponding to the interface:

The neighbor table stores the MAC address corresponding to the destination IP address. The MAC address is the identifier of a Layer 2 Ethernet node. The MAC address is dynamically learned through ARP packets or can be statically specified.

Enter the ARP table:

ARP tables add MAC headers to packets:

Layer 2 packet forwarding:

Layer 2 devices (Bridges, switches, etc.) search for forwarding ports based on destination MAC addresses to implement Layer 2 forwarding.

Physical layer: mainly nic driver, nic chip processing, the data into electrical or optical information or wireless signals;

For details about network layer analysis, see Overview guide for Locating cloud Network Packet Loss Faults

Fairness idea — transmission control

As the Internet becomes larger and larger, computer processing performance improves, and a large number of data packets flow to the network, network congestion will inevitably worsen. When network congestion occurs, data loss, delay increase, throughput decrease, and even “congestion crash” will occur in serious cases.

Congestion control needs to ensure that the network can carry the traffic submitted by users. It is a global problem involving many factors, such as hosts and routers: The Internet uses TCP congestion control algorithm to control the capacity of the system, and the core idea is the active surrender mechanism. When network congestion is found (packet loss or delay increases), the number of packets sent is actively reduced, so that everyone is for everyone and everyone is for everyone, and congestion is finally removed to achieve a win-win situation:

On the terminal system, the size of the congestion control window is usually adjusted according to the information contained in the Acknowledge character (ACK) packet received to control the sending rate of TCP connections. For example: TCP Tahoe, TCP Reno, TCP Vegas, TCP NewReno, TCP BIC, TCP CBIC, BBR and other algorithms;

The congestion control algorithm (AQM-Active Queue Management- Active Queue Management) on the network intermediate devices (routers, firewalls, switches, etc.) usually judges the degree of network congestion control according to the cache Queue length information in the device, and displays or implicitly informs the endpoint of the congestion control information. The endpoint adjusts its transmission rate based on the congestion control information obtained, for example: FIFO, RED(Random Early Detection), ECN(Explicit Congestion Notification), FQ(Fair Queuing) and other algorithms;

Road traffic system is a typical statistical multiplexing packet switching system. In fact, road traffic system has existed for several thousand years before TCP/IP packet switching network. We can use some problems of road traffic system to analyze the problems in TCP/IP network by analogy, such as road congestion and network congestion. Can help us understand the web more easily.

conclusion

Three design ideas of Internet communication are mainly mentioned here:

Distributed thinking – packet switching: Provide enough simple and flexible packet forwarding logic, realize stable forward data message in the system, network nodes forward behavior is not centralized control, without god’s perspective, but in the end network can implement the data forwarding, reference some philosophy reflects the network design, network can be connected to any one by nodes around the relationship between the unknown node (six degrees theory), This may be the underlying theory behind the success of packet switching.

Hierarchical design of network protocol: application layer is highly extensible and applications grow explosively, thanks to the layered model of network;

Fairness thought – transmission control: TCP congestion control makes the network more fair and stable, improve the fault tolerance rate of the system, so that the system can continue normal operation;

Technically, the “revolutionary creation” of connectionless packet switching was a triumph for the Internet. The “systematization and completion” of TCP/IP hierarchical model makes the Internet enter the stage of vigorous commercial development. The Internet is designed to ensure the success of TCP/IP, making it the most important infrastructure in today’s information society.

Some design ideas for dealing with complexity:

  • The bottom layer realizes a set of stable operation mechanism (forwarding mechanism) : for example, the longest matching forwarding mechanism of three-layer routing; Layer 2 MAC address forwarding mechanism (Bridge forwarding mechanism).

  • The upper layer implements configurable policies (various forwarding rules), such as routing table, neighbor table, MAC forwarding table, etc.

  • The classical idea of layered design continues to this day. The idea of adding a layer can realize decoupling or association, which enhances the expansibility of design.

  • The more simple and extensible the design, the better able to cope with future changes;

3 Evolution of computer network technology

The core demands for the continuous development of the Network are:

Network communication scenarios continue to expand

People continue to expand network communication scenarios: LAN, MAN, WAN, wireless network, Internet, mobile Internet, industrial Internet, cloud computing network, Internet of Things, Internet of vehicles and so on;

Network capacity and users have exploded

From Web1.0 and Web2.0 to the development of mobile Internet, social networking, games, short video, live broadcasting and other applications have become popular, and Internet traffic has skyrocketed dozens of times, posing great challenges to the network’s supporting capacity and forcing the network to continue to develop

  • TCP/IP protocols evolve: from TCP congestion control algorithm Reno to BBR, from TCP to QUIC, from HTTP1.0 to HTTP3.0, from IPv4 to IPv6, and so on;

  • Evolution of network architecture: from distributed network to centralized network, SDN emerged.

  • Evolution of mobile network technology: 2G->3G->4G->5G->6G…

  • Evolution of transport network technology: PDH→SDH->WDM->OTN

  • .

Computer network security issues

Computer network security is an important concern in today’s world. Hackers have been using innovative techniques to steal important information. Nowadays, people are using cyberspace for their daily activities, and their dependence on cyberspace is very high. Whether people are being stolen business and technical secrets or financial systems, the threat is deadly.

  • Firewall evolution: Packet firewall -> State firewall -> Application Firewall ->…

  • Evolution of encryption technology: symmetric encryption -> asymmetric encryption -> quantum encryption

  • Security evolution: traditional network security -> mobile network security -> cloud computing network security -> cloud native network security

  • .

The above is part of my understanding of the network, mainly the core idea of understanding, I hope to help you better understand the network, but also for the series of subsequent articles to pave the way, I hope you enjoy

Reference:

The Design Philosophy of The DARPA Internet Protocols

wikipedia.org

Congestion Avoidance and Control

How the Web Is Connected

-END-

Hello, I’m Alex, I work more than eight years, experience huawei — — — — > > foreign company tencent and other companies, currently responsible for tencent network core architecture design and development of cloud, understand all kinds of company promotion route, the interview experience, for large-scale, large flow, high concurrency, high availability, extreme high performance system has the rich experience, good at development background, Network technology and performance optimization, hope to share years of technical research and work experience through the public number, welcome to pay attention to

Geek rebirth

Senior engineer of Goose Factory, core member of Tencent Cloud network, share core cloud computing technology, core network technology, high-performance network server, core Linux kernel technology, rich interview & workplace experience, focus on core knowledge sharing and rebirth together with you.