This article is based on xie Xiren’s “Computer Network (7th edition Publishing House of Electronics Industry)” and the experience of predecessors on the Internet, the network knowledge is more, this article only summarizes the more common knowledge (according to my understanding of continuous update), hope big guy correct.

An overview of the

composition

From the perspective of working mode: the edge part (all hosts connected to the Internet are composed of users directly) and the core part (a large number of networks and routers connected to these networks are composed of services for the edge part)

edges

End system communication mode: client-server mode (C/S) and peer-to-peer mode (P2P)

The core part of the

Router is the key construction to implement packet switching. Its task is to forward the received packets

Three kinds of exchange

  • Circuit switching: A continuous stream of bits of the entire message from source to destination, as if in a pipe.
  • Packet exchange: The whole packet is first transmitted to the adjacent node, all the packets are stored, then searched for the forwarding table and forwarded to the next node.
  • Packet switching: a single packet (part of a packet) is transmitted to a neighboring node, the packet is transmitted to the neighboring node, the packet is stored, the forwarding table is searched, and the packet is forwarded to the next node

category

scope

  • WAN WAN
  • MAN MAN
  • LAN LAN
  • Personal regional network PAN

The user

  • Public network
  • Private networks (VPNS)

Performance indicators

rate

The rate at which a host connected to a computer network transmits bits of data over a digital channel, also called data rate or bit rate. Unit Is b/s, KB /s, Mb/s, Gb/s.

bandwidth

In data communications, the highest rate of data transmitted over a digital channel in b/s, KB /s, Mb/s, or Gb/s.

throughput

That is, the amount of data that passes through a network in unit time, in b/s, Mb/s, etc.

Time delay

  • Transmission delay: Transmission delay = Block length (bit)/Channel bandwidth (bit/second)
  • Propagation delay: Propagation delay = channel length (m)/signal propagation rate over the channel (m/s)
  • Processing delay: storage and forwarding processing time of network nodes
  • Queuing delay: waiting time

Delay-bandwidth product

Latency bandwidth product = propagation latency * bandwidth

Round-trip time

Round-trip Time (RTT) Starts when the sender sends data, and ends when the sender receives data and the receiver acknowledges it

utilization

Network utilization: weighted average value of the channel utilization

The physical layer

role

The role of the physical layer is to mask as much as possible the differences between the hardware devices and the transmission media in the computer network. The main task of the physical layer is to determine the characteristics associated with the interface to transport the media:

  • (1) Mechanical properties: specify the shape and size of the connector used for the interface, the number and arrangement of pins, fixing and locking devices, etc. There are strict standardized rules for the common connectors of various specifications.
  • (2) Electrical characteristics: indicate the voltage range on each line of the interface cable.
  • (3) Functional characteristics: indicate the meaning of voltage of a certain level on a certain line.
  • (4) Process characteristics: indicate the sequence of various possible events for different functions

Data communication system

Source system (source point and sender), transmission system and destination system (receiver and destination)

communication

One-way communication, two-way alternating communication, two-way simultaneous communication

Channel multiplexing

Frequency division multiplexing, time division multiplexing, statistical time division multiplexing, code division multiplexing, WDM

Data link layer

channel

Point-to-point channel (frame, PPP protocol), broadcast channel

Three basic questions

Frame encapsulation (add headers and tails before and after, send from headers), transparent transmission (byte fill or character fill: make possible control characters in data not interpreted as control characters at the receiving end, insert escape characters) and error detection (CYCLIC redundancy check CRC)

Hardware address of the Ethernet MAC layer

A MAC address is an adapter address or adaptation identifier

The network layer

Internet Protocol IP

Form a complete set of protocols

  • Address resolution protocol ARP
  • Internet Control Message Protocol, ICMP
  • Internet Group management protocol IGMP

Routing protocol

  • Internal Gateway Protocol (IGP) : Any information used to exchange routing information within an autonomous system is collectively referred to as the internal gateway protocol. RIP (distance vector method), OSPF (Shortest path First) and IGRP are commonly used on the Internet.
  • External Gateway Protocol (EGP) : The protocol used to transmit network reachability information between two autonomous systems is called the external gateway protocol.

Transport layer

An overview of the

The transport layer provides communication services to the application layer, which provides end-to-end logical communication between application processes

UDP protocol.

Reuse and reuse functions and error detection functions. Connectionless, unreliable, packet-oriented, no congestion control, supports one-to-one, one-to-many, many-to-one, and many-to-many interactive communication, with small header overhead (8 bytes: source port, destination port, length, check and. Two bytes respectively).

TCP protocol

Link-oriented, one can only have two endpoints — point-to-point, reliable, full-duplex communication, byte stream oriented (TCP data units are messages)

The TCP header

  • Two bytes each for the source port and destination port
  • Serial number 4 bytes
  • The confirmation number is 4 bytes
  • The number offset is 4 bits
  • 6 digits
  • URGent URG (URGent) If URG is 1, the URGent pointer field is valid. It tells the system that there is urgent data in this message segment and that it should be sent as soon as possible (equivalent to high-priority data) rather than in the original queued order. For example, a long program has been sent to run on a remote host. But some problems were found that required the program to be canceled, so the user issued an interrupt command from the keyboard. If emergency data is not used, these two characters are stored at the end of the cache that receives TCP. These characters are delivered to the receiver’s application process only after all data has been processed. It wastes a lot of time.
  • When URG is set to 1, the sending application tells the sender’s TCP that it has urgent data to transmit. The sender TCP then inserts the emergency data at the top of the column data, while the data after the emergency data remains normal data. This is used in conjunction with the Urgent Pointer field in the header.
  • ACKnowledgment ACK (ACKnowledgment) The ACKnowledgment number field is valid only when ACK = 1, and invalid when ACK = 0. TCP specifies that ACK must be set to 1 for all transmitted segments after a connection is established.
  • PSH (PuSH) When two application processes communicate interactively, sometimes the application on one side expects to receive a response immediately after typing a command. In this case, TCP can use push operations. In this case, the sender TCP sets PSH to 1 and immediately creates a packet segment to send. After receiving the packet segment with PSH=1, the receiving TCP delivers the packet to the receiving application process as soon as possible. Instead of waiting for the entire cache to fill up and deliver up.
  • ReSeT RST When RST is 1, a serious error occurs in the TCP connection of the table name (for example, due to a host crash or other reasons) and the connection must be released before the transport connection is re-established. RST set to 1 can also be used to reject an invalid message segment or to refuse to open a connection.
  • SYNchronization SYN (SYN) Synchronizes the sequence number when a connection is set up. When SYN=1 and ACK=0, this is a connection request packet segment. If the peer agrees to establish a connection, set SYN=1 and ACK=1 in the response packet segment. Therefore, if the SYN is set to 1, it indicates a connection request or connection acceptance packet.
  • A FINis (FINis, meaning “finish” or “end”) is used to release a connection. If the FIN value is 1, all data on the packet segment is sent and the transport connection is released.
  • Window 2 bytes
  • Check and 2 bytes
  • Emergency pointer 2 bytes
  • Options vary in length, up to 4 bytes. When no option is used, the length of the TCP header is 20 bytes.
  • The TCP header

TCP three-way handshake

  • First handshake: The client sends a SYN packet (seq= X) to the server and enters the SYN_SEND state for confirmation
  • Second handshake: After receiving a SYN packet, the server must acknowledge the client’s SYN(ACK = X +1) and send a SYN packet (SEq = Y). In this case, the server enters the SYN_RECV state
  • Third handshake: After receiving the SYN+ACK packet from the server, the client sends an ACK packet (ACK = Y +1) to the server. After the packet is sent, the client and the server enter the ESTABLISHED state to complete the three-way handshake.

TCP: the three-way handshake

TCP waved four times

  • First wave: The Client sends a FIN to stop data transmission from the Client to the Server, and the Client enters the FIN_WAIT_1 state.
  • Second wave: After receiving a FIN, the Server sends an ACK to the Client. The ACK sequence number is +1 (the same as that for SYN, one FIN occupies one sequence number). The Server enters CLOSE_WAIT state.
  • Third wave: The Server sends a FIN to disable data transfer from the Server to the Client, and the Server enters the LAST_ACK state.
  • Fourth wave: After receiving the FIN, the Client enters the TIME_WAIT state and sends an ACK to the Server to confirm that the FIN number is +1. The Server enters the CLOSED state and waves four times

TCP- Three handshakes and four waves simple to understand

Reliable transport of TCP

Stop waiting protocol

A After the packet is sent, the copy of the sent packet must be retained temporarily, the packet and acknowledgement packet must be numbered, and the retransmission time set by the timeout timer must be longer than the average round-trip time of data transmission in the packet. Pros: Simple. Disadvantages: Low channel utilization

A sliding window in bytes

The groups before the sending window are the groups that have been sent and confirmed. The sending window contains the groups that have been sent but not confirmed and the groups that are allowed to be sent but not yet sent. The groups after the sending window are the groups that are not allowed to be sent in the cache.

  • A’s sending window is set according to B’s receiving window, but A’s sending window is not always the same size as B’s receiving window.
  • TCP usually stores the data that arrives out of order in the receiving window temporarily. After the missing bytes in the byte stream are received, TCP delivers the data to the upper application process in order.
  • TCP requires the receiver to have a cumulative acknowledgment function to reduce transmission overhead, and TCP states that the acknowledgment delay should not exceed 0.5 seconds

Timeout Retransmission time selection

Karm proposed that when calculating weighted average RTTS, the round-trip time sample of the message segment is not used as long as it is retransmitted. Fixed: The RTO of the timeout retransmission time was increased when the segment was not retransmitted

Select confirm SACK

The SACK documentation does not specify how the sender should respond to the SACK, so most of it is retransmission of all unacknowledged blocks.

TCP traffic control

Flow control is realized by sliding window

Flow control is to make the speed of the sender not too fast, the receiver in time to receive

TCP congestion control

congestion

The demand for a resource exceeds the available portion of what the resource can provide

methods

  • Slow start: Gradually increase the sending window from small to large
  • Congestion avoidance: Congestion window CWND slowly increases
  • Fast retransmission: The receiver is required to send repeated acknowledgement immediately after receiving an out-of-order message segment, rather than waiting to send the acknowledgement itself. According to the fast retransmission algorithm, the sender should immediately retransmit the unreceived packet segment as long as it receives three consecutive repeated acknowledgements, rather than waiting for the retransmission timer to expire
  • Fast Recovery: Slow start is used only when THE TCP connection is established and the network times out. When the sender receives three consecutive repeated acknowledgements, the multiplication reduction algorithm is performed to halve the SSthRESH threshold. But then the slow start algorithm is not performed. The sender now assumes that the network may not be congested, considering that it would not receive multiple duplicate acknowledgements if the network were congested. So instead of executing the slow start algorithm, set CWND to ssTHRESH size, and then execute the congestion avoidance algorithm.

The application layer

An overview of the

The application layer is the communication and interaction rule between application processes. The data unit of interaction at the application layer is called message.

Domain Name System (DNS)

An overview of the

A network service that maps network device names to IP addresses. The DNS listens on port 53 of TCP and UDP. [TCP is used for data transmission and UDP is used for query]

Domain structure

Host. Subdomain name (SLD). Top-level domain name (TLD). Root Domain name (root) The DNS server queries the IP address of each domain name on a hierarchical basis

The hierarchical query

Query NS records and A records (IP address) of the top-level domain name Server from the root domain name server. From “top-level domain name server” to “secondary domain name server” NS records and A record (IP address) from “top-level domain name server” to “secondary domain name server” NS records and A record (IP address) from “top-level domain name server” to “secondary domain name server” NS records and A record (IP address) Find the IP address of the host name from the secondary domain name server

NS DNS server that records the domain name of this level

DNS record type

  • A: Address record: Returns the IP address specified by the domain name
  • NS: DNS record: Returns the IP address of the server that stores the next-level domain name information. The record can only be set to a domain name, not an IP address.
  • MX: Mail record: returns the address of the server that receives emails
  • CNAME: Canonical naming record: returns another domain name, that is, the current query domain name is a jump to another domain name
  • PTR: Reverse query record: This record is used to query domain names from IP addresses only, preventing spam

A query

Recursion and iteration: The way we send requests to the local DNS server is recursive query, and the local DNS server requests to other DNS servers is iterative query process

  • The host first performs recursive query to the local DNS server
  • The local DNS server uses iterative query to query a root DNS server
  • The root DNS server tells the local DNS server the IP address of the top-level DNS server that should be queried next time
  • The local DNS server queries the TOP-LEVEL DNS server
  • The TOP-LEVEL DNS server tells the local DNS server to query the IP address of the secondary server
  • The local DNS server queries the secondary server
  • The secondary server tells the local DNS server the IP address of the host being queried
  • The local DNS server finally reports the query results to the host

DNS cache

DNS cache means that after the DNS returns the correct IP address, the system stores the result temporarily. And it sets an expiration date for the cache (say, N hours), within which the system will return the results directly from your computer’s LOCAL DNS cache to you when you visit the site again, without having to ask the DNS server, thereby “speeding up” the resolution of the address.

DNS tools

  • dig
  • host
  • nslookup
  • whois

details

  • Introduction to DNS Principles
  • DNS (Domain name Resolution Protocol) details

FTP

Main functions: Reduce or eliminate file incompatibility in different operating systems. The main process accepts new requests, and slave processes process individual requests. FTP uses two ports: 21 (command port) and 20 (data port)

FTP Protocol Description

HTTP

An overview of the

HTTP is a TCP/ IP-based communication protocol to transfer data, the default port number is 80, using TCP but itself is connectionless, and is stateless, this feature simplifies server design

message

  • Request packet – The client sends a request packet to the server. The first line of the request message is called the request line. The following lines are called the header line. The header line can be followed by an entity body. The request line contains three fields: the method field, the URL field, and the HTTP version field.
  • Response message – the response from the server to the client. The first line of the response message is called the status line, followed by the header line, and finally the entity body. The status line contains three fields: the HTTP version field, the status code, and the phrase.

After the first line there is a blank line, which cannot be omitted. This blank line is used to separate the head from the entity.

Status code

  • 1XX: Indicating message – indicating that the request has been received and processing continues
  • 2xx: Success: The request is successfully received, understood, or accepted
  • 3xx: Redirect – Further action must be taken to complete the request
  • 4XX: Client error – The request has a syntax error or the request cannot be implemented
  • 5xx: Server side error — the server failed to fulfill a valid request

HTTP request methods

HTTP1.0 defines three request methods: GET, POST, and HEAD. HTTP1.1 adds five new request methods: OPTIONS, PUT, DELETE, TRACE, and CONNECT.

  • GET requests the specified page information and returns the entity body.
  • A HEAD is similar to a GET request, except that there is no concrete content in the response returned to retrieve the header
  • POST Submits data to a specified resource for processing requests (such as submitting a form or uploading a file). The data is contained in the request body. POST requests may result in the creation of new resources and/or the modification of existing resources.
  • PUT Transmits data from the client to the server instead of the content of the specified document.
  • DELETE requests the server to DELETE the specified page.
  • CONNECT Reserved in HTTP/1.1 for proxy servers that can pipe connections.
  • OPTIONS allows clients to view server performance.
  • TRACE displays the requests received by the server for testing or diagnosis.

One article on the HTTP protocol is enough

How HTTP works

What happens when the user enters the URL and displays the page

  • DNS resolves the IP address corresponding to the domain name of the URL
  • A TCP connection
  • Sending HTTP requests (three-way handshake)
  • The server processes the request and returns HTTP packets
  • The browser parses the rendered page
  • Connect the end of the

HTTP caching mechanism

Strong cache: If the cache data is not invalid, you can directly use the cache data negotiation cache: The browser sends a request to the server, and the server determines whether to use the local cache

Get and POST

  • GET contains parameters in the URL, and POST passes parameters through the Request body
  • GET is harmless when the browser falls back, while POST resubmits the request.
  • The URL generated by GET can be bookmarked, but not by POST.
  • GET requests are actively cached by browsers, whereas POST requests are not, unless set manually.
  • GET requests can only be url encoded, while POST supports multiple encoding methods.
  • GET request parameters are retained in browser history, while parameters in POST are not.
  • GET requests pass parameters in the URL with length limits, whereas POST does not.
  • GET accepts only ASCII characters for the data type of the argument, while POST has no restrictions.
  • GET is less secure than POST because parameters are exposed directly to the URL and therefore cannot be used to pass sensitive information

The difference between GET and POST basic requests

HTTP1.1

  • HTTP 1.1 contains so much detail and optional parts that it becomes too large.
  • With so many options available, client and server interoperability issues emerge
  • Inadequate use of TCP
  • Queue head blocking causes the maximum number of persistent connections to be reached and the remaining resources to wait for other resource requests to complete.

Spriting, Inlining, Concatenation, and Sharding are all designed to solve these problems

HTTP2

  • Binary: HTTP2 is a binary protocol
  • Binary format: HPACK, HTTP/2 header compression, as the name implies it is a compression format designed for HTTP2 headers
  • Reset-regret: In HTTP2, we can implement this requirement by sending RST_STREAM frames to avoid wasting bandwidth and breaking existing connections.
  • Server push: When a client requests resource X and the server knows it probably needs resource Z as well, the server can proactively push resource Z to the client before the client sends the request
  • Flow control: Each stream on HTTP2 has its own public traffic window that restricts the other end from sending data
  • Allows multiplexing

HTTP2 interpretation

HTTP3

QUIC protocol based on UDP protocol.

  • HTTPS works like a TCP handshake, working on HTTP1, passing the exchange to get the secret key, and then switching to HTTPS.
  • HTTP2 is based on TLS, so HTTP2 works the same way as HTTPS. After the TLS connection is completed, the client server sends an HTTP2 connection confirmation message.
  • HTTP3 first establish a connection, then send HTTP2 extension frame, this frame contains THE IP and port, the browser received the extension frame, use the IP and port, use QUIC to establish a connection, if successful, disconnect HTTP2, upgrade to HTTP3

The HTTP protocol – HTTP3

HTTPS

An overview of the

HTTP+SSL/TLS, that is, HTTP is added to the SSL layer, and HTTPS is based on SS security

With HTTP difference

  • For HTTPS, you need to apply for a certificate from the CA.
  • HTTP is hypertext transmission protocol, information is plaintext transmission; HTTPS is a secure SSL encrypted transport protocol.
  • HTTP and HTTPS use completely different connections and use different ports, the former 80 and the latter 443.
  • HTTP connections are simple and stateless; HTTPS is a network protocol that uses SSL and HTTP to encrypt transmission and authenticate identity. It is more secure than HTTP.

SSL/TLS

handshake

  • The client makes a request to the server containing the protocol version number used, a random number generated, and the encryption method supported by the client.
  • After receiving the request, the server confirms the encryption method used by both parties, gives the server’s certificate, and a random number generated by the server.
  • After confirming that the server certificate is valid, the client generates a new random number, encrypts this random number using the public key in the digital certificate, and sends it to the server. It also provides a hash value for all previous content that the server can verify.
  • The server uses its own private key to decrypt the random number sent by the client. And provide the hash value of all the previous content for the client to verify.
  • The client and the server use the first three random numbers according to the agreed encryption method to generate a dialogue secret key, and the subsequent dialogue process uses this secret key to encrypt information.

encryption

  • Symmetric encryption (also called private key encryption) : An encryption algorithm that uses the same key for encryption and decryption
  • Asymmetric encryption algorithm: Requires two keys: publickey and privatekey. And the encryption key and decryption key come in pairs. Asymmetric encryption algorithms use different keys during encryption and decryption
  • Algorithm: the figures in this paper, is to use single Hash function will need to encrypt plaintext “abstract” into a series of fixed length (128) of the cipher text, the string of cipher text is also known as digital fingerprint, it has a fixed length, and different plaintext into ciphertext, the result is always different, and the same proclaimed in the abstract must be consistent. “Digital digest” is the fundamental reason HTTPS ensures data integrity and tamper-proof.
  • Digital signature technology: it is the application of “asymmetric key encryption and decryption” and “digital abstract”, which encrypts the abstract information with the sender’s private key and transmits it to the receiver with the original text. The receiver can decrypt the encrypted digest only with the sender’s public key, and then use the HASH function to generate a digest of the received text and compare it with the decrypted digest. If they are the same, the received information is complete and has not been modified during transmission. Otherwise, the received information has been modified. Therefore, the digital signature can verify the integrity of the information.
  • Digital certificate: avoid the public key to be compiled, to avoid the target host is engaged in stealing user information such improper behavior

Https protocol details

other

Principle of CDN

Using a variety of cache servers, the cache servers can not be divided into relatively concentrated regions or networks for users to visit. When users visit websites, the global load technology will be used to point to the nearest normal cache server, and the cache server will respond directly

Websocket and ajax

  • Ajax requires the client to initiate a request, and the Websocket server and client can push messages to each other in real time
  • Ajax is characterized by asynchronous interaction and dynamic updating of Web pages. Therefore, Ajax is applicable to web applications with more interaction and frequent data reading: form data verification with Ajax, data fetching on demand, and automatic updating of pages
  • Application scenarios of WebSocket: social applications, stock fund applications, location-based applications, online education applications

What you should know about Ajax and WebSockets

OSI seven-layer model and TCP/IP four-layer model

  • Application layer, presentation layer, session layer, transport layer, network layer, data link layer, physical layer
  • Application layer, transportation layer, Internet layer, network interface layer

Cross domain

When the protocol, domain name, or port of a URL request is different from the current page URL, it is called cross-domain

  • Jsonp: The use of Script is not restricted by the same origin policy. Disadvantages: Only get mode, vulnerable to XSS attack
  • Modify document.domain across subdomains: document.domain = ‘test.com’. Disadvantages: same level domain name; Same protocol; The same port
  • Proxy cross-domain request: the front end sends a request to the server through the proxy to request the required server resources. Disadvantage: Additional proxy servers are required
  • Html5 postMessage method: allows scripts from different sources to communicate asynchronously in a limited way, which can achieve cross-text, multi-window, cross-domain messaging. Disadvantages: Browser version requirements, some browsers need to configure the release of cross-domain restrictions
  • Html5 WebSocket: This protocol enables full-duplex communication between the browser and the server, allowing cross-domain requests. Disadvantages: Certain versions of the browser require that the server support webSocket
  • Document. XXX + IFrame: Iframe is the browser non-homologous tag, load content transfer, transfer to the current page of the property, disadvantages: page property value has a size limit
  • Cross-origin Resource Sharing (CORS) : When using XMLHttpRequest to send a request, if the browser finds that the same Origin policy is violated, it automatically adds a request header Origin. Access-control-allow-origin: access-Control-origin: access-Control-allow-origin: access-Control-allow-origin: access-Control-allow-origin: access-Control-allow-origin The browser checks whether the access-Control-allow-origin value in the response is the same as the current address and processes the response only after the match is successful. Otherwise, the cookie is ignored. The browser version has certain requirements
  • What is cross-domain? Cross-domain solutions
  • What is cross-domain? How to implement (detailed explanation)

token

  • The client frequently requests data from the server, and the server frequently queries and compares the user name and password in the database to determine whether the user name and password are correct or not, and gives corresponding prompts. In this context, the Token comes into being.
  • Token is a string generated by the server as a Token for the client to request. After the first login, the server generates a Token and returns the Token to the client. The client only needs to bring the Token to request data without the need to bring the user name and password.
  • Token’s purpose is to reduce the strain on the server, reduce frequent queries to the database, and make the server more robust.
  • Token usage: The device ID/MAC address is used as the Token, and the session value is used as the Token

What is a token