OSI seven-layer model (idealized network model)

  1. The application layer
  2. The presentation layer
  3. The session layer actually belongs to the application layer
  4. Transport layer TCP UDP protocol
  5. Network layer router
  6. Switch NIC at the data link layer
  7. Physical layer Physical device Network cable Optical fiber

Actual five layers

  • In actual applications, the session layer, presentation layer, and application layer are combined into an application layer
  1. The data that the user wants to pass is packaged HTTP DNS
  2. Transport layer users who want to pass large packets will split and label the large packetsThe serial number The port numberUsed to transmit TCP and UDP packetsPeriod of
  3. The network layer wraps the data again with IP addresses and if the data at the transport layer is still very large the network layer subcontracts the IP protocol againpackage
  4. The data link layer is again wrapped asframe
  5. The physical layer these two layers are concerned with hardware

Five-tier model (collectively known as TCP/IP protocol cluster)

  • What can be called protocols are above the data link layer, so there are protocols at the network layer, transmission layer and application layer
  • The function of a protocol is to encapsulate data in accordance with a specification, and then transmit the data (protocol is the encapsulation + transmission of data).
  • The network layer
    • IP protocol addressing
    • ARP Obtains MAC Addresses from IP addresses (in the LOCAL Area Network)
  • Transport layer: TCP and UDP
  • Application layer: HTTP hypertext transfer protocol, DNS domain name resolution to IP address, FTP file transfer protocol

DNS protocol

  • DNS server A server that translates domain names to IP addresses

TCP/IP protocol

  • Addressing Locate the final device by IP address

ARP protocol

  • How to find the MAC address from the IP address
    • ARP: The purpose is to find the MAC address from the IP address (in the LOCAL area network)
    • It broadcasts to each of them, and it responds to the IP address it’s looking for, and it returns the MAC address
    • ARP will be cached and broadcast cache is IP: MAC
    • The switch caches MAC: ports

TCP protocol

  • Transmission control protocol is characterized by reliability and low transmission efficiency
  • TCP provides the full-duplex service, that is, data can be transmitted in both directions at the same time
  • The data is transferred unordered across the network, and the receiver needs an algorithm to restore the original order after receiving the data
  • Three-way handshake
    • Can I call you unsolicited
    • Yeah, can I call you, too
    • The connection can be established successfully
  • Four times to wave
    • Let’s break up
    • Reply to receiving a breakup message
    • Okay, Break up
    • Okay, so that’s it
  • summary
    • TCP is duplex, so three handshakes are required to ensure agreement (establishing a connection wastes performance)
    • When disconnecting, the other party needs to reply immediately (ACK) when sending (FIN), but may not be able to close immediately (there are unfinished data and some operations ready to disconnect), so wait until it is confirmed that it can close (FIN).
  • The sliding window
    • Window size is in bytes
    • When establishing a TCP connection, the receiver tells its window how many bytes of data it can receive
    • The purpose of sliding window is to realize packet loss and retransmission, and effectively send data to the receiver, and you can know how many packets are sent each time, so as to achieve traffic control
    • So the core of sliding Windows isControl flow
    • What TCP does: It makes a sliding window during transmission, constantly negotiates the size of the sliding window, stops transmission if the value is full, keeps sending probe packets, and then transfers data if the window size is available. In order, the sliding window is moved back after a successful data is received, and the window size is modified according to the result of negotiation
  • Slow start, congestion avoidance, fast retransmission, fast recovery
    • It’s also used for flow control
    • Slow start timeout
    • Fast retransmission does not require three timeout confirmations immediately retransmission

http

  • The development course
    • HTTP /0.9 No request header and request body contents are transmitted in HTML using ASCii character streams
    • HTTP /1.0 adds request headers and response headers for multiple types of data transfer
    • HTTP / 1.1
      • Persistent Connection Multiple HTTP requests can be transmitted over a TCP connection
      • In piped mode, a maximum of six TCP persistent connections are maintained for each domain name
      • Introduce client cookie mechanism, security mechanism and so on
    • HTTP / 2.0
      • Using the multiplexing mechanism, one domain name uses one TCP long link
      • The head of compression
      • Server push
    • HTTP / 3.0 based on UDP
  • What happens when you visit a web page
    • When a client sends a request it sends a domain name and the domain name is sent to the DNS server
    • The DNS resolves an IP address to the client
    • HTTP transmits data. HTTP generates HTTP request packets for the target Web server
    • Transfer to TCP: To facilitate communication, HTTP request packets are divided into packet segments by serial number and each packet segment is reliably transmitted to the other party (resending the packet if the packet is discarded or timed out).
    • (TCP itself has no transmission capacity, how to rely on the network layer) IP protocol: address and route to find each other’s address, through the MAC address to find the next station, non-stop transfer
    • After receiving a TCP packet segment from the peer, the peer reorganizes the packet segment according to the original sequence number
    • HTTP parses the passed data and processes the content
    • After processing, add some response information, also use TCP/IP communication protocol to send back to the user

HTTP is a protocol that does not save state and uses cookies to manage state

To prevent the establishment and disconnection of TCP connections each time, keep-alive is used

Before sending a request, you had to wait and receive a response before sending the next one. Now it’s all piped

The HTTP application

  • HTTP also encapsulates some information and transport
  • curl -v www.baidu.com
  • HTTP is divided into two parts (sending request client) req -> (receiving request server) res
  • The request is divided into three parts (all three parts can transmit data)
    • The request row goes through the method path (the data transferred is limited by the URL size limit)
      • Such as GET/version number
    • Request headers that put some custom information conventions should not be too large
    • The data transferred by the request body
  • The response is also divided into three parts
    • Response line
      • For example, the version number 200 is OK
    • Response headers
    • Response body
  • RestFul style
    • GET/POST/PUT/SELETE/OPTIONS
    • OPTIONS requests represent cross-domain access requests that may be pretested if the other party confirms that it can send a real request
    • OPTIONS requests can only be sent in the state of complex requests (get, POST are simple requests, and complex requests with custom headers).
    • OPTIONS allows you to define the sending interval
  • Common status codes
    • You can set it up but there are some conventions between the browser and the server
    • The 1xx server receives the message and waits for the browser to do something later with the Websocket
    • 2 xx success
    • 3xx redirection cache
    • 4XX client error (browser parameters, or server cannot parse client parameters)
    • 5XX Server error
    • 200 Request succeeded
    • 204 Request successful but no response
    • Section 206 Content segmentation request
    • 301 302 Permanent and temporary redirects
    • 304 the cache
    • 400 The client request is incorrect
    • 401 Permission Problem The current user does not log in and has no permission to view the video
    • 403 logged in but still didn’t have permission
    • 404 can’t find
    • 405 The server only supports GET and POST, but the server cannot find this method if it sends a PUT request
    • 500 Request service resolution error cannot complete response
    • Contents received during 502 service period are invalid
    • 503 The load balancer is down
const http = require("http");

// The following two expressions are equivalent
const server = http.createServer(function(req,res) {
    console.log("request")}); server.on('request'.function(req,res) {
    console.log("request")});let port = 3000;

server.listen(3000.function()  {
    console.log('server listen on 3000')});// Listening error
server.on("error".function(err){
    if(err.code === 'EADDRINUSE') { // Indicates that the port number is occupied
        server.listen(++port);// If the callback succeeds, it will go to the corresponding callback above, and print server Listen on 3000}})Copy the code
  • HTTP module encapsulation of REQ and RES
const server = http.createServer(function(req, res) {
    // req is a readable stream

    / / request
    console.log(req.method); / / caps
    console.log(req.url); 
    console.log(req.httpVersion);

    / / request header
    console.log(req.headers); // Unified Node processing is all lowercase

    / / request body
    const arr = [];
    req.on("data".function(data) {
        console.log(data);
        arr.push(data);
    });
    req.on("end".function(){
        const data = Buffer.concat(arr).toString();
        console.log("end", data); })})const server = http.createServer(function(req, res) {
    // res is a writable stream

    res.statusCode = 202;
    res.statusMessage = "my 202";
    res.setHeader("token"."ok");
    res.write("1");
    res.end("2");
});
Copy the code

Implementing static services

const http = require("http");

// Url composition: Protocol ://(username: password) Domain: port number/resource path? Query parameter #hash
const url = require("url");
// The argument to true converts query to object form
// const {pathname,query} = url.parse("http://username:[email protected]:3000/xxx?a=1#hash", true);
// pathname => /xxx

const mime = require("mime");

const server = http.createServer((req,res) = > {
  const {pathname} = url.parse(req.url, true);
  // Read the file by path /public/index.html
  const filePath = path.join(__dirname, pathname); // Get the absolute path
  fs.readFile(filePath, function(err, data) {
    if(err) {
      res.statusCode = 404;
      return res.end('not found');
    }
    
      
    // If the browser does not display to the encoding browser, it will be garbled
    // So it is possible that the browser does not encode the data and the server needs to encode the data
    res.setHeader("Content-Type", mime.getType(filePath) + "; charset=utf8"); res.end(data); })}); server.listen(3000);

Copy the code
  • Write a static server
    • Official: NPM I HTTP-server-g can start a service locally
    • Use: command line hs or HTTP-server to find the public folder by default
// package.json
{
    "name": "zhuhaha-server-listen"."version":"1.0.0"."description": ""."main":"1.http.js"."bin": {"zsl": "./bin/www"}}/* This package is unusable by default. To test it globally using NPM link will generate a soft chain called zhuha-server-listen and a soft chain called ZSL */WWW file: #! /usr/bin/env node re-npm link --foreCopy the code
  • The server should support: usecommanderThe module
    • –port Changes the port number
    • –directory Specifies which directory to base against
    • — help command
    • –version
    • –useage
  • Implementation of template engine
    • with + new Function
    • Ejs.renderfile (path.resolve(__dirname, 'tmppl.html '), {arr:[1,2,3,4,5]}, {async: true}).then(data =>.console.log (data))
  • Implementation of command line configuration — Commander module
// WWW does some configuration on the command line
const {program} = require("commander");
const version = require(".. /package.json").version;
const Server = require(".. /src/main.js");
const config  =  {
    'port': {
        option: '-p,--port <n>'.description: 'set server port'.default: 8080.usage: 'zsl --port <n>'
    },
    'directory': {
        option: '-d,--directory <n>'.description: 'set server directory'.default: process.cwd(),
        usage: 'zsl -d D:'
    }
}

program.version(version)  / / version
.name("zsl")
.usage("[options]")

const usages =  [];
Object.entries(config).forEach(([key, value]) = > {
    usages.push(value.usage);
    program.option(value.option,  value.description,  value.default)  // Configure options
})

program.on("--help".function() { // An example
    console.log("\nExamples:")
    usages.forEach(usage= > console.log("" + usage))
})

program.parse(process.argv); // Parse command line arguments

let ops = program.opts(); // Get the parameter object
console.log(ops);

let server = new Server(ops);
server.start(); // Start the service


Copy the code
  • Implementation of template engine
// self-implementing template engine
async function renderFile(filePath, data) {
    let tmplStr = await fs.readFile(filePath, 'utf8');
    let myTemplate = `let str = ''\r\n`;
    myTemplate += 'with(obj){'
    myTemplate +=  'str+=`'
    tmplStr = tmplStr.replace(/ < % = (. *?) %>/g.function() {
        return '${' + arguments[1] + '} ';
    })
    myTemplate += tmplStr.replace(/<%(.*)%>/g.function() {
        return '`\r\n' + arguments[1] + '\r\nstr+=`'
    })
    myTemplate += '`\r\n return str \r\n}';
    let fn = new Function('obj', myTemplate);

    return fn(data);
}

async function render(tmplStr, data) {
    let myTemplate = `let str = ''\r\n`;
    myTemplate += 'with(obj){'
    myTemplate +=  'str+=`'
    tmplStr = tmplStr.replace(/ < % = (. *?) %>/g.function() {
        return '${' + arguments[1] + '} ';
    })
    myTemplate += tmplStr.replace(/<%(.*)%>/g.function() {
        return '`\r\n' + arguments[1] + '\r\nstr+=`'
    })
    myTemplate += '`\r\n return str \r\n}';
    let fn = new Function('obj', myTemplate);

    return fn(data);
}
Copy the code
  • template
// tmpl.html<! DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta  name="viewport" content="width=device-width, Initial-scale =1.0"> <title>Document</title> </head> <body> <% dirs.foreach (item =>%> <a href="<%=item.url%>"><%=item.dir%></a> <%)%> </body> </html>Copy the code
  • Setting up static HTTP services
// src/main.js
const http = require("http");
const fs = require("fs").promises;
const path = require("path");
const url = require("url");
const mime = require("mime");
const os = require("os");
const chalk = require("chalk");
const zlib = require("zlib");
const crypto. = require('crypto');
const {createReadStream, readFileSync} = require("fs");

let address = Object.values(os.networkInterfaces()).flat().find(item= > item.family == 'IPv4').address;

const template = readFileSync(path.resolve(__dirname, 'tmpl.html'), 'utf8');

module.exports = class MyServer{
    constructor(opts={}) {
        this.port = opts.port;
        this.directory = opts.directory;
        this.address = address;
        this.template = template;
    }

    handleRequest = async (req, res) => {
        console.log(this);
        // When the request comes in, you need to monitor the path to see if the path is a file. If it is a file, return the file directly. If it is not a file, read the directory in the file
        let {pathname} = url.parse(req.url);
        pathname = decodeURIComponent(pathname); // Translate the Chinese in the path to ensure normal display
        let filePath = path.join(this.directory, pathname); // The absolute path of the file is found in the current execution directory
        try{
            let statObj =  await fs.stat(filePath);
            if(statObj.idDirectory()) {
                const dirs = await fs.readdir(filePath);
                // Use template engine to do this
                let content = await render(template, {
                    dirs: dirs.map(dir= > ({
                        url: path.join(pathname, dir),
                        dir
                    }))
                });
                res.setHeader("Content-Type"."text/html; charset=utf-8");
                res.end(content);
            }else{
                / / file
                this.sendFile(req,res,statObj,filePath); }}catch(e) {
            // An error occurs if the file does not exist
            this.sendError(res, e); }}gzip(req,res){
        // Compress before sending
        Accept-encoding: gzip, deflate, br
        // Node supports only gaip and Deflate by default
        // The server will say in the response header: Content-encoding: Gzip is compressed by gzip
        let encoding = req.headers['accept-encoding'];
        let zip;
        if(encoding){
            // Indicates that compression is supported
            let ways = encoding.split("，");
            for(let i = 0; i < ways.length; i++){
                if(ways[i] === 'gzip'){
                    res.setHeader('content-encoding'.'gzip'); 
                    zip = zlib.createGzip();
                    break;
                }else if(ways[i] === 'deflate'){
                    res.setHeader('content-encoding'.'deflate');
                    zip = zlib.createDeflate();
                    break; }}}return zip;
    }
    cache(req, res, statObj, filePath){
        Before sending the file, you can request that the file not be accessed again for a certain amount of time
        // Only for referenced resources resources accessed for the first time are not set
        // that is, pages accessed directly by the cache cannot be cached
        // res.setHeader('Cache-Control', 'max-age=10'); Cache-control takes precedence over Expires
        // res.setHeader('Expires', new Date(Date.now()+10*1000).toGMTString()); // Set an expiration timestamp for earlier browsers
        
        res.setHeader('cache-control'.'no-cache'); // The file will still be cached. No -store is to ask the server every time, no cache
        // Negotiate cache
        // Some files may not change after 10 seconds. If the file does not change, continue to search the cache
        // The last-modified server tells the browser when the file was last modified
        // if-modified-since the next time the browser visits it
        console.log(req.headers['if-modified-since']); // The value will not be printed until the server is set
        const ifModifiedSince = req.headers['if-modified-since'];
        const ctime = statObj.ctime.toGMTString();
        res.setHeader('last-modified'.)// if(ifModifiedSince ! == ctime){
            // return false;
        // } 
        // Depending on the last modification time, it may change after the change but the content does not change, or if it changes several times within 1s, it will not monitor because the cache time is in seconds
        // tag generates unique identifiers based on the content
        // md5 digest is irreversible, so the result of the digest is the same, so you can use the database to deduce the original result
        // The server provides etag and the browser provides if-none-match
        const ifNoneMatch = req.headers['if-none-match'];
        // Real development will not read the file, usually use a certain part of the file or calculate the size of the total length of the file
        const etag = crypto.createHash('md5').update(fs.readFileSync(filePath)).digest('base64');
        res.setHeader('ETag', etag);
        if(ifNoneMatch ! = etag){return false;
        }
        return true;
    }
    
    sendFile(req,res,statObj,filePath) {
        / / cache
        if(this.cache(req,res,statObj,filePath)) {
            res.statusCode = 304;
            return res.end();
        }
        res.setHeader("Content-Type", (mime.getType(filePath) || "text/plain") + "; charset=utf-8")
        // The file is returned directly to the browser without being compressed to read
        // createReadStream(filePath).pipe(res); // Asynchronous method
        let zip = this.gzip(req,res);
        if(zip){ // Support compression compression
            createReadStream(filePath).pipe(zip).pipe(res);
        }else{ createReadStream(filePath).pipe(res); }}sendError(res, e) { 
        console.log('err', e);
        res.statusCode = 404;
        res.end("not found");
    }
    start() {
        console.log('start ' + this.port  + "" + this.directory);
        const server = http.createServer(this.handleRequest);
        server.listen(this.port, () = > {
            console.log(`${chalk.yellow('starting up http-server, seerving:')}` + this.directory);
            consolee.log(`  http://${address}:${chalk.green(this.port)}`);
            consolee.log(`  http://127.0.0.1:${chalk.green(this.port)}`)}}}/** If it is a file, the file content will be read by stream and returned to the browser. If it is a folder, the template engine will render the file or subdirectory under the folder, and return to the browser. When clicking on the corresponding subdirectory, the corresponding content will be displayed
// Now the static service is basically implemented

Copy the code

The compression

  • The main compression method for Gzip compression is substitution. The higher the repetition rate, the more effective the compression is
const zlib = require("zlib");
zlib.createGzip // Stream the way read point operation point
zlib.gzip()  // Non-stream mode


zlib.gzip(fs.readFileSync(path.resolve(__dirname, '1.txt')), function(err,data){
    fs.writeFileSync('1.txt.gz', data)
})
Copy the code
  • Four ways to flow
    • Readable stream on(‘data’) on(‘end’)
    • Writable stream Write End
    • Duplex flow can read and write
    • The conversion stream transforms data, such as compression and encryption
// Standard input reads the user's input
process.stdin.on('data'.function(chunk){ / / to read
    console.log(chunk); // Will listen to the user's input on the command line,
})

process.stdin.on('data'.function(chunk){
    process.stdout.write(chunk); // Can write equivalent to console.log
})

// The two strings can be abbreviated as:
process.stdin.pipe(process.stdout)


// If you want to convert input from lowercase to uppercase, you use the transform stream
const {Transform} = require('stream');
class MyTransform extends Transform{
    _transfrom(chunk, encoding, clearBuffer){ // The arguments are the same as writable streams
        this.push(chunk.toString().toUpperCase()); clearBuffer(); }}let transform = new MyTransform();
process.stdin.pipe(transform).pipe(process.stdout);
Copy the code
  • Compress files in a static service before sending them to the browser

The cache

  • Mandatory cache
    • Cache-control: ‘Max – age = 10’
    • Expires Sets the expiration timestamp
  • Negotiate the cache
    • The last-Modified server returns the last-Modified server to the client
    • If-modified-since This is used when the client sends a request
  • The characteristics of the md5
    • The irreversible
    • The same content produces the same result
    • The results are completely different depending on the content
    • The length after the summary is the same
    • Const r = crypto.createHash('md5').update(' content ').digest('base64'); Creating an MD5 Digest
    • The server provides eTAG
    • Browsers provide if-none-match